Быстрый отслеживания объектов - Robot Computer Vision
Я хотела моя робота , чтобы иметь возможность следить за объектом и следовать им. Первое, что я хотел сделать, это дать роботу способность следовать объекта с его головы камеры. Голова камера установлена на пан-наклон сервосистемы и, следовательно, способны перемещаться влево и вправо, вверх и вниз (как показано на рисунке ниже).
Моей второй целью отслеживания объекта было сделать робота погоня за объектом, так же, как собака погони мяч, брошенный своим хозяином. Такое отслеживание довольно трудно - он будет использовать голове камеры слежения на предыдущем шаге, и объединить его с остальными датчиками робота следовать за объектом.
Как?
Для этого я буду использовать несколько основных обработки изображений \ алгоритмов компьютерного зрения. Я собираюсь использовать OpenCV библиотеки. OpenCV, как это следует из названия, является открытым исходным кодом компьютерного видения библиотеки первоначально
разработанная Intel. Это кросс-платформенный (я использовал его как на компьютере и на ARM основана BeagleBoard ). OpenCV довольно легко использовать, если у вас есть базовые знания по обработке изображений.
Первый объект я хотел трек был обычными цветными оранжевый мяч.
Цвет слежения
Фильтр только оранжевый цвет с картинки. Чтобы сделать это, я преобразовал изображение в HSV цветового пространства , а затем использовал cvInRange фильтр дважды, чтобы фильтр оранжевого цвета.
В первом рисунке вы можете увидеть видео пара преобразуется в цветовое пространство HSV, а во второй картинке вы можете видеть результат красный цвет фильтрации (все цвета, кроме красного были отфильтрованы).
(Python-код)
#declare necessary objects hsv_frame = cvCreateImage(size, IPL_DEPTH_8U, 3) thresholded = cvCreateImage(size, IPL_DEPTH_8U, 1) thresholded2 = cvCreateImage(size, IPL_DEPTH_8U, 1) hsv_min = cvScalar(0, 50, 170, 0) hsv_max = cvScalar(10, 180, 256, 0) hsv_min2 = cvScalar(170, 50, 170, 0) hsv_max2 = cvScalar(256, 180, 256, 0) # convert to HSV for color matching # as hue wraps around, we need to match it in 2 parts and OR together cvCvtColor(frame, hsv_frame, CV_BGR2HSV) cvInRangeS(hsv_frame, hsv_min, hsv_max, thresholded) cvInRangeS(hsv_frame, hsv_min2, hsv_max2, thresholded2) cvOr(thresholded, thresholded2, thresholded)
Shape based tracking
Then, I used Hough transform to detect the shape of a circle. Before applying the hough transform I smoothed the image because it seems to improve the results.
# pre-smoothing improves Hough detector cvSmooth(thresholded, thresholded, CV_GAUSSIAN, 9, 9) circles = cvHoughCircles(thresholded, storage, CV_HOUGH_GRADIENT, 2, thresholded.height/4, 100, 40, 20, 200)
The result
The result is pretty impressive. The code functions very well and detects the ball under most circumstances (i.e. the ball is far from the camera, close to the camera, slow movement, fast movement, etc).
You can see the result in this video:
Full python sources including servo movement
/***************************************************************************************** * Name : Fast object tracking using the OpenCV library * * Author : Lior Chen <chen.lior@gmail.com> * * Notice : Copyright (c) Jun 2010, Lior Chen, All Rights Reserved * * : * * Site : http://www.lirtex.com * * WebPage : http://www.lirtex.com/robotics/fast-object-tracking-robot-computer-vision * * : * * Version : 1.0 * * Notes : By default this code will open the first connected camera. * * : In order to change to another camera, change * * : CvCapture* capture = cvCaptureFromCAM( 0 ); to 1,2,3, etc. * * : Also, the code is currently configured to tracking RED objects. * * : This can be changed by changing the hsv_min and hsv_max vectors * * : * * License : This program is free software: you can redistribute it and/or modify * * : it under the terms of the GNU General Public License as published by * * : the Free Software Foundation, either version 3 of the License, or * * : (at your option) any later version. * * : * * : This program is distributed in the hope that it will be useful, * * : but WITHOUT ANY WARRANTY; without even the implied warranty of * * : MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * * : GNU General Public License for more details. * * : * * : You should have received a copy of the GNU General Public License * * : along with this program. If not, see <http://www.gnu.org/licenses/> * ******************************************************************************************/ #!/usr/bin/python # -*- coding: utf-8 -*- from opencv.cv import * from opencv.highgui import * from threading import Thread #import serial class RobotVision: cvSize size cvImage hsv_frame, thresholded, thresholded2 cvScalar hsv_min, hsv_max, hsv_min2, hsv_max2 cvCapture capture; def InitBallTracking(): globals size, hsv_frame, thresholded, thresholded2, hsv_min, hsv_max, hsv_min2, hsv_max2, capture print "Initializing ball Tracking" size = cvSize(640, 480) hsv_frame = cvCreateImage(size, IPL_DEPTH_8U, 3) thresholded = cvCreateImage(size, IPL_DEPTH_8U, 1) thresholded2 = cvCreateImage(size, IPL_DEPTH_8U, 1) hsv_min = cvScalar(0, 50, 170, 0) hsv_max = cvScalar(10, 180, 256, 0) hsv_min2 = cvScalar(170, 50, 170, 0) hsv_max2 = cvScalar(256, 180, 256, 0) storage = cvCreateMemStorage(0) # start capturing form webcam capture = cvCreateCameraCapture(-1) if not capture: print "Could not open webcam" sys.exit(1) #CV windows cvNamedWindow( "Camera", CV_WINDOW_AUTOSIZE ); def TrackBall(i): t = Thread(target=TrackBallThread, args=(i,)) t.start() def TrackBallThread(num_of_balls): globals size, hsv_frame, thresholded, thresholded2, hsv_min, hsv_max, hsv_min2, hsv_max2, capture while 1: # get a frame from the webcam frame = cvQueryFrame(capture) if frame is not None: # convert to HSV for color matching # as hue wraps around, we need to match it in 2 parts and OR together cvCvtColor(frame, hsv_frame, CV_BGR2HSV) cvInRangeS(hsv_frame, hsv_min, hsv_max, thresholded) cvInRangeS(hsv_frame, hsv_min2, hsv_max2, thresholded2) cvOr(thresholded, thresholded2, thresholded) # pre-smoothing improves Hough detector cvSmooth(thresholded, thresholded, CV_GAUSSIAN, 9, 9) circles = cvHoughCircles(thresholded, storage, CV_HOUGH_GRADIENT, 2, thresholded.height/4, 100, 40, 20, 200) # find largest circle maxRadius = 0 x = 0 y = 0 found = False for i in range(circles.total): circle = circles[i] if circle[2] > maxRadius: found = True maxRadius = circle[2] x = circle[0] y = circle[1] cvShowImage( "Camera", frame ); if found: print "ball detected at position:",x, ",", y, " with radius:", maxRadius if x > 420: # need to pan right servoPos += 5 servoPos = min(140, servoPos) servo(2, servoPos) elif x < 220: servoPos -= 5 servoPos = max(40, servoPos) servo(2, servoPos) print "servo position:", servoPos else: print "no ball"
Sample Sources
This c++ code takes a video stream from an attached video camera, looks for an orange ball inside the stream, and prints the coordinates of the ball.
Three “debug” windows are shown to clarify the process: 1) the video capture. 2) the stream after the conversion to HSV, 3) the stream after conversion to HSV, color-filtering, and Hough transform.
/*****************************************************************************************
* Name : Fast object tracking using the OpenCV library *
* Author : Lior Chen <chen.lior@gmail.com> *
* Notice : Copyright (c) Jun 2010, Lior Chen, All Rights Reserved *
* : *
* Site : http://www.lirtex.com *
* WebPage : http://www.lirtex.com/robotics/fast-object-tracking-robot-computer-vision *
* : *
* Version : 1.0 *
* Notes : By default this code will open the first connected camera. *
* : In order to change to another camera, change *
* : CvCapture* capture = cvCaptureFromCAM( 0 ); to 1,2,3, etc. *
* : Also, the code is currently configured to tracking RED objects. *
* : This can be changed by changing the hsv_min and hsv_max vectors *
* : *
* License : This program is free software: you can redistribute it and/or modify *
* : it under the terms of the GNU General Public License as published by *
* : the Free Software Foundation, either version 3 of the License, or *
* : (at your option) any later version. *
* : *
* : This program is distributed in the hope that it will be useful, *
* : but WITHOUT ANY WARRANTY; without even the implied warranty of *
* : MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the *
* : GNU General Public License for more details. *
* : *
* : You should have received a copy of the GNU General Public License *
* : along with this program. If not, see <http://www.gnu.org/licenses/> *
******************************************************************************************/
#include <opencv/cvaux.h>
#include <opencv/highgui.h>
#include <opencv/cxcore.h>
#include <stdio.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <assert.h>
#include <math.h>
#include <float.h>
#include <limits.h>
#include <time.h>
#include <ctype.h>
int main(int argc, char* argv[])
{
// Default capture size - 640x480
CvSize size = cvSize(640,480);
// Open capture device. 0 is /dev/video0, 1 is /dev/video1, etc.
CvCapture* capture = cvCaptureFromCAM( 0 );
if( !capture )
{
fprintf( stderr, "ERROR: capture is NULL \n" );
getchar();
return -1;
}
// Create a window in which the captured images will be presented
cvNamedWindow( "Camera", CV_WINDOW_AUTOSIZE );
cvNamedWindow( "HSV", CV_WINDOW_AUTOSIZE );
cvNamedWindow( "EdgeDetection", CV_WINDOW_AUTOSIZE );
// Detect a red ball
CvScalar hsv_min = cvScalar(150, 84, 130, 0);
CvScalar hsv_max = cvScalar(358, 256, 255, 0);
IplImage * hsv_frame = cvCreateImage(size, IPL_DEPTH_8U, 3);
IplImage* thresholded = cvCreateImage(size, IPL_DEPTH_8U, 1);
while( 1 )
{
// Get one frame
IplImage* frame = cvQueryFrame( capture );
if( !frame )
{
fprintf( stderr, "ERROR: frame is null...\n" );
getchar();
break;
}
// Covert color space to HSV as it is much easier to filter colors in the HSV color-space.
cvCvtColor(frame, hsv_frame, CV_BGR2HSV);
// Filter out colors which are out of range.
cvInRangeS(hsv_frame, hsv_min, hsv_max, thresholded);
// Memory for hough circles
CvMemStorage* storage = cvCreateMemStorage(0);
// hough detector works better with some smoothing of the image
cvSmooth( thresholded, thresholded, CV_GAUSSIAN, 9, 9 );
CvSeq* circles = cvHoughCircles(thresholded, storage, CV_HOUGH_GRADIENT, 2,
thresholded->height/4, 100, 50, 10, 400);
for (int i = 0; i < circles->total; i++)
{
float* p = (float*)cvGetSeqElem( circles, i );
printf("Ball! x=%f y=%f r=%f\n\r",p[0],p[1],p[2] );
cvCircle( frame, cvPoint(cvRound(p[0]),cvRound(p[1])),
3, CV_RGB(0,255,0), -1, 8, 0 );
cvCircle( frame, cvPoint(cvRound(p[0]),cvRound(p[1])),
cvRound(p[2]), CV_RGB(255,0,0), 3, 8, 0 );
}
cvShowImage( "Camera", frame ); // Original stream with detected ball overlay
cvShowImage( "HSV", hsv_frame); // Original stream in the HSV color space
cvShowImage( "After Color Filtering", thresholded ); // The stream after color filtering
cvReleaseMemStorage(&storage);
// Do not release the frame!
//If ESC key pressed, Key=0x10001B under OpenCV 0.9.7(linux version),
//remove higher bits using AND operator
if( (cvWaitKey(10) & 255) == 27 ) break;
}
// Release the capture device housekeeping
cvReleaseCapture( &capture );
cvDestroyWindow( "mywindow" );
return 0;
}
,,,,i wanted to ask {CvCapture* capture = cvCaptureFromCAM( 0 );
