\(\mathbb{M}\)orphological-\(\mathbb{S}\)ymmetry-Equivariant
\(\mathbb{H}\)eterogeneous \(\mathbb{G}\)raph \(\mathbb{N}\)eural \(\mathbb{N}\)etwork
for Robotic Dynamics Learning

Fengze Xie*1, Sizhe Wei*2, Yue Song1, Yisong Yue1, Lu Gan2
1 Caltech
2 Georgia Tech
*Indicates Equal Contribution
Paper Code arXiv
SOLO C2 Symmetry

A1 robot demonstrating \(\mathbb{C}_2\) symmetry properties

Mini Cheetah K4 Symmetry

Mini-Cheetah exhibiting \(\mathbb{K}_4\) symmetry properties

Method Overview

Method Overview

Overview of our MorphSym-HGNN architecture incorporating morphological symmetries for robotic dynamics learning.

Abstract

We present a morphological-symmetry-equivariant heterogeneous graph neural network, namely MS-HGNN, for robotic dynamics learning, that integrates robotic kinematic structures and morphological symmetries into a single graph network. These structural priors are embedded into the learning architecture as constraints, ensuring high generalizability, sample and model efficiency. The proposed MS-HGNN is a versatile and general architecture that is applicable to various multi-body dynamic systems and a wide range of dynamics learning problems. We formally prove the morphological-symmetry-equivariant property of our MS-HGNN and validate its effectiveness across multiple quadruped robot learning problems using both real-world and simulated data.

Compare with SOTA Methods on Classification Task

Figure 2
Figure 2: Contact state detection results visualization. Left: F1 score for each leg, averaged F1 score, and 16-state contact state accuracy, averaged over 4 random runs. The number of parameters for each method is also provided. Right: Averaged F1 score for models trained using various numbers of training samples. Our MS-HGNN ( \(\mathbb{C}_2\) & \(\mathbb{K}_4\) ) achieve around 0.9 averaged F1-score trained with only 5% of the entire training set.
Table 1: Contact state detection results on the real-world Mini-Cheetah dataset. Numerical results of Figure 2 (left).
Model Sym. Leg-LF \(F_1\) \(\uparrow\) Leg-LH \(F_1\) \(\uparrow\) Leg-RF \(F_1\) \(\uparrow\) Leg-RH \(F_1\) \(\uparrow\) Accuracy \(\uparrow\) \(F_1\) Score \(\uparrow\)
CNN - 0.771 ± 0.013 0.899 ± 0.003 0.884 ± 0.014 0.891 ± 0.024 0.731 ± 0.013 0.861 ± 0.004
CNN-Aug \(\mathbb{C}_2\) 0.854 ± 0.009 0.896 ± 0.022 0.835 ± 0.015 0.906 ± 0.013 0.778 ± 0.019 0.873 ± 0.007
ECNN \(\mathbb{C}_2\) 0.884 ± 0.012 0.887 ± 0.010 0.853 ± 0.011 0.860 ± 0.016 0.788 ± 0.029 0.871 ± 0.011
MI-HGNN \(\mathbb{S}_4\) 0.932 ± 0.006 0.936 ± 0.010 0.927 ± 0.003 0.928 ± 0.005 0.870 ± 0.010 0.931 ± 0.005
MS-HGNN \(\mathbb{C}_2\) 0.928 ± 0.013 0.933 ± 0.011 0.913 ± 0.016 0.937 ± 0.010 0.856 ± 0.013 0.929 ± 0.009
MS-HGNN \(\mathbb{K}_4\) 0.936 ± 0.008 0.944 ± 0.006 0.930 ± 0.011 0.948 ± 0.006 0.875 ± 0.012 0.939 ± 0.006
Table 2: Sample efficiency results on the real-world Mini-Cheetah contact dataset. The dataset comprises 634.6K training and validation samples. Numerical results of Figure 2 (right).
Training Samples (%)
Model Sym 2.50 5.00 10.00 15.00 21.25 42.50 63.75 85.00
CNN - 0.745 0.794 0.831 0.802 0.811 0.840 0.850 0.836
CNN-Aug \(\mathbb{C}_2\) 0.764 0.851 0.827 0.859 0.844 0.829 0.839 0.881
ECNN \(\mathbb{C}_2\) 0.840 0.841 0.851 0.843 0.867 0.877 0.785 0.881
MI-HGNN \(\mathbb{S}_4 (G)\) 0.872 0.908 0.926 0.930 0.937 0.940 0.932 0.931
MS-HGNN \(\mathbb{C}_2\) 0.760 0.893 0.910 0.923 0.926 0.939 0.935 0.939
MS-HGNN \(\mathbb{K}_4\) 0.869 0.897 0.913 0.922 0.919 0.939 0.935 0.942

Compare with SOTA Methods on Regression Task (GRF Estimation)

Experimental Results
Figue 1(b). Ground reaction force estimation test RMSE on simulated A1 dataset.
Table 3. Estimation performance (RMSE) on simulated dataset. Numerical results of Figure 1(b).
Test Sequence Test RMSE \(\downarrow\)
Z-GRF (1D) GRF (3D)
MI-HGNN MS-HGNN (\(\mathbb{C}_2\)) MI-HGNN MS-HGNN (\(\mathbb{C}_2\))
Unseen Friction 8.089 ± 0.102 7.850 ± 0.154 6.437 ± 0.055 6.355 ± 0.050
Unseen Speed 9.787 ± 0.111 9.733 ± 0.142 7.887 ± 0.064 7.721 ± 0.048
Unseen Terrain 8.826 ± 0.144 8.685 ± 0.136 7.332 ± 0.076 7.208 ± 0.047
Unseen All 10.245 ± 0.168 10.137 ± 0.084 8.708 ± 0.052 8.630 ± 0.097
Total 9.035 ± 0.116 8.899 ± 0.079 7.388 ± 0.056 7.268 ± 0.032

Compare with SOTA Methods on Regression Task (CoM Estimation)

Figure 3
Figure 3: Centroidal momentum estimation results on the synthetic Solo dataset. Left: The test linear, angular cosine similarity, and MSE of each model's prediction, averaged over 4 random runs. Right: The linear cosine similarity for models of different sizes. Our MS-HGNN ( \(\mathbb{C}_2\) & \(\mathbb{K}_4\) ) methods exhibit superior model efficiency without overfitting.
Table 4. Centroidal momentum estimation results on the synthetic Solo dataset. Numerical results of Figure 3 (left).
Model \(\mathbb{G}\) Lin. Cos. Sim. \(\uparrow\) Ang. Cos. Sim. \(\uparrow\) Test MSE \(\downarrow\) \(\mathbb{G}\) Lin. Cos. Sim. \(\uparrow\) Ang. Cos. Sim. \(\uparrow\) Test MSE \(\downarrow\)
MLP - 0.9617 ± 0.0036 0.9523 ± 0.0032 0.0499 ± 0.0037 - 0.9617 ± 0.0036 0.9523 ± 0.0032 0.0499 ± 0.0037
MLP-Aug \(\mathbb{C}_2\) 0.9639 ± 0.0026 0.9535 ± 0.0029 0.0478 ± 0.0020 \(\mathbb{K}_4\) 0.9647 ± 0.0023 0.9549 ± 0.0023 0.0472 ± 0.0014
EMLP \(\mathbb{C}_2\) 0.9610 ± 0.0039 0.9528 ± 0.0051 0.0503 ± 0.0053 \(\mathbb{K}_4\) 0.9673 ± 0.0045 0.9580 ± 0.0032 0.0435 ± 0.0048
MS-HGNN \(\mathbb{C}_2\) 0.9903 ± 0.0001 0.9804 ± 0.0015 0.0161 ± 0.0006 \(\mathbb{K}_4\) 0.9877 ± 0.0007 0.9799 ± 0.0010 0.0189 ± 0.0007
Table 5. Model size and performance comparison on the synthetic Solo dataset. Numerical results of Figure 3 (right).
Method # of Parameters / Lin. Cos. Sim. \(\uparrow\)
MLP 10,310 / 0.9147 36,998 / 0.9631 - / - 139,526 / 0.9737 - / - - / - 541,190 / 0.9643 - / -
MLP-Aug (\(\mathbb{C}_2\)) 10,310 / 0.9170 36,998 / 0.9660 - / - 139,526 / 0.9802 - / - - / - 541,190 / 0.9839 - / -
EMLP (\(\mathbb{C}_2\)) - / - 36,992 / 0.9640 - / - 139,520 / 0.9843 - / - - / - 541,184 / 0.9865 - / -
MS-HGNN (\(\mathbb{C}_2\)) 13,478 / 0.9448 26,150 / 0.9558 52,550 / 0.9746 102,470 / 0.9870 207,494 / 0.9903 405,638 / 0.9945 464,838 / 0.9940 824,582 / 0.9959
MI-HGNN (\(\mathbb{S}_4\)) 12,934 / 0.8864 25,478 / 0.9136 50,438 / 0.9213 100,102 / 0.9297 199,174 / 0.9275 223,878 / 0.9325 396,806 / 0.9319 791,558 / 0.9276
MLP-Aug (\(\mathbb{K}_4\)) 10,310 / 0.9199 36,998 / 0.9644 - / - - / - - / - - / - 541,190 / 0.9843 - / -
EMLP (\(\mathbb{K}_4\)) - / - 36,992 / 0.9718 - / - 139,520 / 0.9868 - / - - / - 541,184 / 0.9910 - / -
MS-HGNN (\(\mathbb{K}_4\)) 11,366 / 0.9240 21,926 / 0.9505 44,230 / 0.9675 85,830 / 0.9854 174,470 / 0.9875 339,590 / 0.9935 390,726 / 0.9915 692,998 / 0.9936

BibTeX

@misc{xie2024morphologicalsymmetryequivariantheterogeneousgraphneural,
        title={Morphological-Symmetry-Equivariant Heterogeneous Graph Neural Network for Robotic Dynamics Learning}, 
        author={Fengze Xie and Sizhe Wei and Yue Song and Yisong Yue and Lu Gan},
        year={2024},
        eprint={2412.01297},
        archivePrefix={arXiv},
        primaryClass={cs.RO},
        url={https://arxiv.org/abs/2412.01297}, 
      }