Creating CNN object detection component in robocomp

Design Specifications:

This component will be online and wait for image to be passed in form of RGB pixel values. The basic idea is to have a asynchronously triggerable remote procedure call. So that when the robot reaches any frontier for example a table where it can find objects, the object detection module is ready. Also this setting provides opportunity to load CNN apriori which is a time consuming process. Thus the object detection requires computing just the forward pass of the network and could be done at 0.3-0.4 Hz that is near real-time. ISDL interface file for the component: ============ module RoboCompobjectDetectionCNN {

struct BoundingBox
{
   float x;
   float y;
   float width;
   float height;    
};

struct ColorRGB
{
    byte red;
    byte green;
    byte blue;
};

struct Label
{
    string name;
    float belive;
    BoundingBox bb;
};

sequence<ColorRGB> ColorSeq;

sequence<Label> ResultList;

interface objectDetectionCNN
{
	void getLabelsFromImage(ColorSeq image, int rows, int cols, out ResultList result);
	
}; };

CSDL interface file for the component:

import “/robocomp/interfaces/IDSLs/objectDetectionCNN.idsl”; Component objectDetectionCNN { Communications { implements objectDetectionCNN;

};
language Cpp;
gui Qt(QWidget); };

Specific worker compute method:

void SpecificWorker::compute() { ///initialize caffe model only once ///read the model file and other parameters from “etc/caffe_config”

if(first) { first=false;

::google::InitGoogleLogging("objectDetectionCNN");
read_caffe_config("etc/caffe_config",model_file, trained_file, mean_file, label_file);
classifier= new Classifier(model_file, trained_file, mean_file, label_file);

}

}

Specific worker get detections:

Pipeline:

1. Convert ColorSeq to opencv::Mat.
2. Now use top-hat and biletral filter to get object proposals.
3. Once we have object proposal create a region of interest and pass it caffe classifier.
4. Return the detections: bounding boxes, labels and classid.

Harit Pandya