RoboComp Logo

A simple robotics framework.

GSoC’21 RoboComp project: Sign language recognition

17th July 2021

Current process:

I have finished to build and test pytorch code for:

  • BodyHandDetector model -> ONNX, tensorRT.
  • ImageBasedGestureRecognition -> ONNX, tensorRT.
  • PoseBasedGestureRecognition -> ONNX, tensorRT.

Besides, the robocomp component for all of them have already coded:

  • BodyHandDetector + BodyHandDetectorClient (Tested + still improving)
  • ImageBasedGestureRecognition (Testing)
  • PoseBasedGestureRecognition (Debugging)

I will soon merge the code of these component into Robocomp lib after GSoC phase 1.

Some problems I find out in Robocomp library:

  • when import default components and generate code py robocompdsl for cdsl for python. Some library can not be imported, for example: rc… lib => add this line sys.path.append(‘/opt/robocomp/python/’) into /usr/local/bin/robocompdsl.py , line 16

  • The install process may face bugs, if not install robocomp lib in home/user_name/ folder. Because, the base code of robocomp lib use static links. And if change the clone folder, the install process may fail.

Current issue:

  • I have some problem with Qcore thread in my component, and it slow down my inference process in pose detection.
  • Need a strategy to do real-time inference for image based and pose based

Works in 2nd phase:

  • debug all remain components.
  • improve the inference speed.
  • code the unsupervised component.