I found when you play the shape-key animation it causes the models mesh to move far from its origin to the point that it disappears.
Not sure if this is an issue with the way I am exporting the model, a bug in panda3d, or a bug in panda3d-gltf.
I made a bug report here: