Development of an AI-driven robotic manipulation framework

Abstract

This paper introduces a manipulator control system that leverages state-of-the-art artificial intelligence (AI), advanced robotics, and the latest computer vision techniques to enable intuitive and universal robotic control across different robotic manipulators. The intuitive user command interpretation is deployed in the system using ChatGPT API. The vision capabilities of the system rely on YOLOv7 for object detection, Semi-Global Block Matching (SGBM) for object localization, and Principal Component Analysis (PCA) for object orientation. Moreover, MoveIt is utilized for motion planning, and the simulation environment is powered by Gazebo. These components are seamlessly coordinated by a custom state machine integrating into a ROS-based framework. This system allows users to control manipulators completing complex tasks by commands without expert knowledge of robotics. Additionally, a cross-platform kinematics solver architecture is introduced to assess inverse kinematics algorithms for manipulators. Compared to the traditional kinematics solver implemented as ROS plugins, this architecture integrates the sophisticated mathematical capabilities of Matlab with the feasibility of ROS.