天津智能网站建设多少钱,免费海报模板网站,网站首页布局分析,房管局网上查询系统文章目录 一、说明二、什么是Lucas-Kanade 方法三、Lucas-Kanade 原理四、代码实现4.1 第 1 步#xff1a;用户在第一帧绘制一个矩形4.2 第 2 步#xff1a;从图像中提取关键点4.3 第 3 步#xff1a;跟踪每一帧的关键点 一、说明
本文针对基于光流法的目标追踪进行叙述用户在第一帧绘制一个矩形4.2 第 2 步从图像中提取关键点4.3 第 3 步跟踪每一帧的关键点 一、说明
本文针对基于光流法的目标追踪进行叙述首先介绍Lucas-Kanade 方法的引进以及基本推导然后演示如何实现光流法的运动跟踪。并以OpenCV实现一个基本项目。
二、什么是Lucas-Kanade 方法
在计算机视觉领域Lucas-Kanade 方法是 Bruce D. Lucas 和Takeo Kanade开发的一种广泛使用的光流估计差分方法。该方法假设所考虑像素局部邻域中的光流基本恒定并根据最小二乘准则求解该邻域中所有像素的基本光流方程。
通过结合来自多个邻近像素的信息Lucas-Kanade 方法通常可以解决光流方程固有的模糊性。与逐点方法相比该方法对图像噪声的敏感度也较低。另一方面由于它是一种纯局部方法因此无法提供图像均匀区域内部的流信息。
三、Lucas-Kanade 原理
在理论上初始时间为 t 0 t_0 t0 时刻经历过 Δ t \Delta t Δt时段后点p会移动到另一个位置 p ′ p′ p′ 并且 p ′ p′ p′ 本身和周围都有着与p相似的亮度值。朴素的LK光流法是直接用灰度值代替RGB作为亮度。根据上面的描述对于点p而言假设p 的坐标值是( x , y )有 I ( x , y , t ) I ( x Δ x , y Δ y , t Δ t ) I(x, y, t) I(x\Delta x,y\Delta y, t\Delta t) I(x,y,t)I(xΔx,yΔy,tΔt)
根据泰勒公式在这里把x 、y 看做是t 的函数把公式(1)看做单变量t 的等式只需对t进行展开 I ( x , y , t ) I ( x , y , t ) ∂ I ∂ x ∂ x ∂ t ∂ I ∂ y ∂ y ∂ t ∂ I ∂ t o ( Δ t ) I(x,y,t)I(x,y,t)\frac{∂I} {∂x}\frac{∂x}{∂t}\frac{∂I} {∂y}\frac{∂y}{∂t}\frac{∂I} {∂t}o(Δt) I(x,y,t)I(x,y,t)∂x∂I∂t∂x∂y∂I∂t∂y∂t∂Io(Δt) 对于一个像素区域 I x ( q 1 ) V x I y ( q 1 ) V x − I t ( q 1 ) I x ( q 2 ) V x I y ( q 2 ) V x − I t ( q 2 ) . . . I x ( q n ) V x I y ( q n ) V x − I t ( q n ) I_x(q_1)V_xI_y(q_1)V_x-I_t(q_1)\\I_x(q_2)V_xI_y(q_2)V_x-I_t(q_2)\\...\\I_x(q_n)V_xI_y(q_n)V_x-I_t(q_n) Ix(q1)VxIy(q1)Vx−It(q1)Ix(q2)VxIy(q2)Vx−It(q2)...Ix(qn)VxIy(qn)Vx−It(qn)
在这里 q 1 , q 2 , . . . q n q_1,q_2,...q_n q1,q2,...qn是窗口内点的标号 I x ( q i ) I_x(q_i) Ix(qi), I y ( q i ) I_y(q_i) Iy(qi), I t ( q i ) I_t(q_i) It(qi)是图像的灰度偏导数 这些方程可以写成矩阵形式 A v b Avb Avb 这个系统的方程多于未知数因此它通常是过度确定的。Lucas-Kanade方法通过最小二乘原理得到折衷解。也就是说它解决了2×2系统: 或 因此
四、代码实现
4.1 第 1 步用户在第一帧绘制一个矩形
# Path to video
video_pathvideos/bicycle1.mp4
video cv2.VideoCapture(video_path)# read only the first frame for drawing a rectangle for the desired object
ret,frame video.read()# I am giving big random numbers for x_min and y_min because if you initialize them as zeros whatever coordinate you go minimum will be zero
x_min,y_min,x_max,y_max36000,36000,0,0def coordinat_chooser(event,x,y,flags,param):global go , x_min , y_min, x_max , y_max# when you click the right button, it will provide coordinates for variablesif eventcv2.EVENT_RBUTTONDOWN:# if current coordinate of x lower than the x_min it will be new x_min , same rules apply for y_min x_minmin(x,x_min) y_minmin(y,y_min)# if current coordinate of x higher than the x_max it will be new x_max , same rules apply for y_maxx_maxmax(x,x_max)y_maxmax(y,y_max)# draw rectanglecv2.rectangle(frame,(x_min,y_min),(x_max,y_max),(0,255,0),1)if you didnt like your rectangle (maybe if you made some misclicks), reset the coordinates with the middle button of your mouseif you press the middle button of your mouse coordinates will reset and you can give a new 2-point pair for your rectangleif eventcv2.EVENT_MBUTTONDOWN:print(reset coordinate data)x_min,y_min,x_max,y_max36000,36000,0,0cv2.namedWindow(coordinate_screen)
# Set mouse handler for the specified window, in this case, coordinate_screen window
cv2.setMouseCallback(coordinate_screen,coordinat_chooser)while True:cv2.imshow(coordinate_screen,frame) # show only first frame k cv2.waitKey(5) 0xFF # after drawing rectangle press ESC if k 27:cv2.destroyAllWindows()breakcv2.destroyAllWindows()4.2 第 2 步从图像中提取关键点
# take region of interest ( take inside of rectangle )
roi_imageframe[y_min:y_max,x_min:x_max]# convert roi to grayscale
roi_graycv2.cvtColor(roi_image,cv2.COLOR_BGR2GRAY) # Params for corner detection
feature_params dict(maxCorners20, # We want only one featurequalityLevel0.2, # Quality threshold minDistance7, # Max distance between corners, not important in this case because we only use 1 cornerblockSize7)first_gray cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)# Harris Corner detection
points cv2.goodFeaturesToTrack(first_gray, maskNone, **feature_params)# Filter the detected points to find one within the bounding box
for point in points:x, y point.ravel()if y_min y y_max and x_min x x_max:selected_point pointbreak# If a point is found, convert it to the correct shape
if selected_point is not None:p0 np.array([selected_point], dtypenp.float32)plt.imshow(roi_gray,cmapgray)将从此图像中提取关键点
4.3 第 3 步跟踪每一帧的关键点
############################ Parameters ####################################
winSize -- size of the search window at each pyramid level
Smaller windows can more precisely track small, detailed features -- slow or subtle movements and where fine detail tracking is crucial.
Larger windows is better for larger displacements between frames , more robust to noise and small variations in pixel intensity -- require more computations
# Parameters for Lucas-Kanade optical flow
lk_params dict(winSize(7, 7), # Window sizemaxLevel2, # Number of pyramid levelscriteria(cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03))############################ Algorithm ##################################### Read video
cap cv2.VideoCapture(video_path)# Take first frame and find corners in it
ret, old_frame cap.read()width old_frame.shape[1]
height old_frame.shape[0]# Create a mask image for drawing purposes
mask np.zeros_like(old_frame)frame_count 0
start_time time.time()old_gray first_graywhile True:ret, frame cap.read()if not ret:breakframe_gray cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)if p0 is not None:# Calculate optical flowp1, st, err cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params) good_new p1[st 1] # st1 means found pointgood_old p0[st 1]if len(good_new) 0:# Calculate movementa, b good_new[0].ravel()c, d good_old[0].ravel()# Draw the tracksmask cv2.line(mask, (int(a), int(b)), (int(c), int(d)), (0, 255, 0), 2)frame cv2.circle(frame, (int(a), int(b)), 5, (0, 255, 0), -1)img cv2.add(frame, mask)# Calculate and display FPSelapsed_time time.time() - start_timefps frame_count / elapsed_time if elapsed_time 0 else 0cv2.putText(img, fFPS: {fps:.2f}, (width - 200, 30), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 0, 0), 2, cv2.LINE_AA)cv2.imshow(frame, img)# Update previous frame and pointsold_gray frame_gray.copy()p0 good_new.reshape(-1, 1, 2)else:p0 None# Check if the tracked point is out of frameif not (25 a width):p0 None # Reset p0 to None to detect new feature in the next iterationselected_point_distance 0 # Reset selected point distance when new point is detected# Redetect features if necessaryif p0 is None:p0 cv2.goodFeaturesToTrack(frame_gray, maskNone, **feature_params)mask np.zeros_like(frame)selected_point_distance0frame_count 1k cv2.waitKey(25)if k 27:breakcv2.destroyAllWindows()
cap.release()
结果
