Navigation and obstacle avoidance are critical for the successful completion of UAV tasks. However，traditional autonomous flight systems face limitations in complex environments，prompting researchers to explore alternative frameworks such as deep reinforcement learning (DRL). This paper proposes a novel DRL⁃based autonomous control algorithm for UAVs，which integrates the Deep Deterministic Policy Gradient (DDPG) algorithm to self⁃learn an optimal Proportional⁃Integral⁃Derivative (PID) controller.The performance of the proposed algorithm is evaluated through simulations in the Gazebo 3D robotic simulator to validate its effectiveness under complex conditions. Results indicate that the proposed method outperforms numerous existing methods in dynamic environments，particularly in terms of improved stability， faster response speed，and higher success rates.

This paper proposes a model?free control protocol design method based on off?policy reinforcement learning for solving the optimal consensus problem of heterogeneous multi?agent systems with leaders. The dynamic expression of local neighborhood error is complicated for the heterogeneous multi?agent systems because of its different system state matrices. Compared with the existing solution of designing observer for distributed control of multi?agent system, the method of solving global neighborhood error state expression proposed in this paper reduces the complexity of calculation. Firstly, the dynamic expression of global neighborhood error of multi?agent system constructed from augmented variables is established. Secondly, the coupled Bellman equation and HJB equation are obtained through the value function of quadratic form. Then, the Nash equilibrium solution of the multi?agent optimal consensus is obtained by solving the optimal solution of the coupled HJB equation, and the Nash equilibrium proof is given. Thirdly, an off?policy Q?learning algorithm is proposed to learn the Nash equilibrium solution of the multi?agent optimal consensus. Then, the proposed algorithm is implemented by using the critic neural network structure and gradient descent method. Finally, a simulation example is given to verify the effectiveness of the proposed algorithm.