In this study, we introduce a novel approach, the Multi-Stream Adaptive Graph Convolutional Neural Network (MS-AAGCN), for skeleton-based action recognition. It addresses previous issues by dynamically learning graph topology and incorporating second-order skeleton data information. The model’s adaptability enhances generality and accuracy, outperforming existing methods on NTU-RGBD and Kinetics-Skeleton datasets.