How can I speed up collision tests?

Hi, I need to build a simulator using Panda3D in which many characters (around 100) must collide with themselves and walls. I’ve noticed that if I disable the collisions with the walls everything runs smoothly (perhaps because the collision solid is a sphere), but when I activate it the program really slows down.

Is there any technique I could use to enhance the collision testing?