Dynamic Task Control Method of a Flexible Manipulator Using a Deep Recurrent Neural Network

Abstract: The flexible body has advantages over the rigid body in terms of environmental contact thanks to its underactuation. On the other hand, when applying conventional control methods to realize dynamic tasks with the flexible body, there are two difficulties: accurate modeling of the flexible body and the derivation of intermediate postures to achieve the tasks. Learning-based methods are considered to be more effective than accurate modeling, but they require explicit intermediate postures. To solve these two difficulties at the same time, we developed a real-time task control method with a deep recurrent neural network named Dynamic Task Execution Network (DTXNET), which acquires the relationship among the control command, robot state including image information, and task state. Once the network is trained, only the target event and its timing are needed to realize a given task. To demonstrate the effectiveness of our method, we applied it to the task of Wadaiko (traditional Japanese drum) drumming as an example, and verified the best configuration of DTXNET.

• The paper introduces DTXNET, a deep recurrent neural network that enhances dynamic task control by integrating multi-modal sensor data.
• It demonstrates how combining joint state and image data significantly reduces errors and refines task execution in flexible manipulators.
• Experimental results from a Wadaiko drumming task validate DTXNET’s potential for real-time control in complex, underactuated robotic systems.

Insights into Dynamic Task Control of Flexible Manipulators Using DTXNET

This paper presents an investigation into the dynamic control of flexible manipulators utilizing a deep recurrent neural network, dubbed the Dynamic Task Execution Network (DTXNET). The authors seek to address the dual challenge of accurately modeling flexible bodies and deriving intermediate postures for task accomplishment—tasks particularly challenging due to the inherent underactuation and flexibility of these systems. By leveraging DTXNET, the research proposes a learning-based control framework that mitigates the necessity for precise modeling and enables real-time task execution.

Overview and Architecture

DTXNET is developed as a deep learning architecture designed to manage the complexities associated with flexible manipulators. It employs a recurrent structure with Long Short-Term Memory (LSTM) units to leverage temporal dependencies, which is a suitable choice given the dynamic nature of the tasks addressed. The framework is versatile, allowing for predictions of task states several steps into the future, thus facilitating the control of tasks that include temporally sparse events.

The architecture offers six configurations, primarily differentiated by whether the robot state—a composite of joint angles, velocities, torques, and image data—is predicted. It also varies depending on whether the input states incorporated are actuator states or image data. The study emphasizes the integration of both Joint State and Image as inputs, yielding superior prediction performance due to the complementary information they provide.

Experimental Evaluation

The authors demonstrate the efficacy of DTXNET through its application to a task involving Wadaiko drumming—a dynamic and intricate task given the manipulation of sound production. This task serves as a challenging testbed due to the inherent flexibility of the manipulator and the underactuated system which necessitates precise timing and positioning for successful execution.

Findings reveal that when DTXNET incorporates both actuator and image data, there's a marked improvement in task execution. In comparison to traditional control methods, as represented through a baseline random control scheme, DTXNET consistently delivers reduced errors in task states. Of note, the model configured to predict robot states as well as task states (Type $3^+$) showed the best performance, indicating the advantage of maintaining a comprehensive dataset of both joint statistics and visual data.

Implications and Future Directions

The research provides significant insights into the utility of deep learning models for the control of flexible manipulators. By effectively decoupling task execution from exact modeling and manual intervention, DTXNET opens pathways for sophisticated automation applications where physical interactions are complex and difficult to predict accurately with traditional methodologies.

There are substantial implications for future robotic systems, especially those requiring intricate environmental interactions, such as soft robotics or systems with compliant components. DTXNET's flexibility in handling various inputs and outputs suggests potential broad applicability across different robotic platforms and tasks.

Future developments may focus on enhancing the model's capabilities to perform multi-task sequences, as well as extending the framework to handle other forms of sensory inputs beyond visual and joint data, such as tactile and auditory feedback. The evolution of this framework could significantly advance the field of robotics by providing more adaptable, intelligent systems capable of learning and responding autonomously in real time to dynamic, complex environments.