Dynamic game programming

I’m planning to port a C#/DirectX project to Panda3D.
I’m especially interested in avoiding time wastes, and my performances in writing code for this project in C#/DirectX were very good, thanks to the Visual Studio IDE that usually guess what I’m going to write, and detects mistakes before I build.

I miss such an IDE for python (i find Eclipse PyDev and SPE not so good), and I’m talking about IDEs because I include the way of writing code as a parameter of productivity.
In this sense, I was more productive with C# and VS8 than with Python and PyDev. But I’m sure Python could wins if i was able to program dynamically.

For the moment, I load a text-console (a window with python syntax highlighting based on the Scintilla engine), that executes dynamically some code (and eventually redefines a function). The goal is to be able to change my program without having to reload the engine (it takes at least 2 seconds on my computer, which is too much if you’re trying to tune a parameter). I’m not 100% satisfied with that for some reason :

• I have problems with the “exec” context (“self” is defined as global)
• Unable to “reset” and cancel all the previous modifications
• IDE features missing (no auto-completion nor syntax checking)

I’m looking for some ways to improve the Panda dev using the python dynamic features. Some ideas :

• The engine is loaded and runs. I’m programming with my favorite IDE, and when I save the file, the engine detects it, “reset all” (it needs to be clarified) and runs the file as if it was the first time

• “States”, like the history in Photoshop. I often implement it by commenting/uncommenting out code blocks. Reloading a previous state would reload everything except the Panda engine. A “state” would be composed of a piece of code stored in a text file, and a memory serialization. Looks like a savegame.

I’m not familiar with game designing/programming, but i think the designer/programmer as to feel and act like a gamer. Then I think you cannot disconnect completely the fact of programming an gaming. How do you mix both environments ?
If some parameters has to be changed during the game for tuning, a common way is to enable their modification with visual controls like sliders, buttons etc. in what we could call a “debug panel”. It’s also frequently done with a console that allows execution of predefined commands.

What about programming while you’re playing ?

In the VR Studio, we program directly into a running Python all the time. We’re old school here–we use Emacs as our “IDE”, and Python is just running in one of the buffers. You can type Python commands interactively at the command prompt, changing the values of variables as you like, moving things around onscreen, calling functions, whatever–and immediately see the results in your Panda window.

You can also modify a class, rewrite a method, write a new class, or whatever, and paste the new class into your running Python process, and immediately see the results of your new code. If it crashes (i.e. generates a Python exception), no problem; fix it and paste it again.

I basically start the game up once, at the beginning of the day, and just keep it running. Then I can do major development into the game without ever shutting it down.

I’d give you advice on how to do this in Emacs, but from your description of the features you’re looking for in an IDE, I suspect you wouldn’t be happy with Emacs. I know some of our users have made similar real-time development environments work in some of the other, more modern Python IDE’s. They may be able to give you some pointers there.

David

Okay, that’s the way I want to go, I’m sure you save a lot of time.

I am kind of new school (actually, no school) : use Eclipse under Windows while you might be running Emacs under debian or maybe gentoo, with fluxbox as your window manager

Anyway, I’m just looking for the highest productivity since my real job doesn’t let me lot of free time (I’m an adept of desktop switching while working) so I’m open to any new environment.
I’ve only used Emacs a few time and I’m sure I could have tweaked it a lot to fit my requirements.

What about technical details.

I’m very interested in running the game once a day and modify classes dynamically. What I do now is running a wxwidgets console :

    #### Console ####
    def __consoleExec(self, evt):
        cmd = self.__consoleScte.Text.strip()+'\n'
        cmd = cmd.replace('\r\n','\n')
        file = open("console.py","w")
        file.writelines(cmd)
        file.close()
        try :
            exec cmd
        except Exception, ex :
            print "Exception :", ex
   
    def __consoleThread(self, task):
        while 1:
            while self.__console.Pending():
                self.__console.Dispatch()
                return Task.cont
            time.sleep(.01)

But that doesn’t let me change really the source code, it just executes something when I click on “Execute”. I currently don’t keep a trace of all that has been executed, so I can’t directly paste the window code into my main source code.

Could you detail how you “reload” some class ? I mean, do you reset everything except the engine that is loaded when you import Panda and type run() or do you only redefine functions by executing def method(…) ?
You say you change your code in Emacs, so I understand that you don’t really use the python console but change directly your source code…? Or maybe are you trying stuffs in the console and then paste them in the source ?

As I’m not so familiar with dynamic languages, so I can’t see how to do what you’re talking about.

Thanks for answering.

Python has a command prompt built-in. If you never call run(), or if you call taskMgr.stop(), or if you send an interrupt sequence (e.g. control-C) to the running Python, it will drop you to the Python prompt to let you type in commands. Then you can call run() again to resume execution.

This is how we run Python within Emacs. The M-x py-shell command in Emacs will start a Python process and put its output in a buffer, which becomes the Python command prompt. That’s 90% of it.

If you take the file in direct/src/directscripts/python-mode.el and copy it into your emacs/site-lisp directory, you will pick up a custom version of Python mode that will add the other 10%. Here’s what we have added:

• Pressing enter when the process is running will send a KeyboardInterrupt to the running Python shell, and drop you back to the prompt. (If for some reason this doesn’t work, C-c C-c does the same thing.)

• Pressing C-d (that is, Emacs-speak for control-D) at the command prompt automatically types run() for you. The clock is resumed from the point at which you paused earlier, so it is as if no time has elapsed while you were at the prompt.

• Pressing C-c C-v with your cursor inside a class definition will repaste the class and rebind all of its methods. This is the magic part, and it works 95% of the time. What this is actually doing is reimporting the class definition into your Python process, similar to the Python reload() call. However, the reload() call is incomplete, because all it does is create a new class definition; it doesn’t change any current instances of your class definition. So the C-c C-v walks through the entire executing Python namespace, looking for any instances of the old class definition, and replaces them immediately with the new class definition. It also replaces method pointers that have been saved. (It only works 95% of the time, because it doesn’t catch some cases where you saved a method pointer directly. For instance, I’m not sure if it can find the Func() intervals and replace them out.)

Though we have set this up in Emacs, it is possible to do these tricks with any IDE. Actually, the last step is really the icing on the cake; the key thing is just to have access to the Python prompt. As long as you can type Python commands directly, especially the import command and reload() method, you’re golden.

You could do this same trick with your wx console. What the Emacs code is actually doing when you repaste a class is writing the new code to a temporary file, then issuing an import command to Python, and then running the Finder.rebindClass() function to replace all the already-existant pointers. You can see this by looking at the python-mode.el script.

David

Thanks a lot for this complete answer. I’ll give feedback soon.

I agree. Python is horrendous to programme, awful IDE’s.

It would be GREAT if you could use Panda3D inside Java, creating jar libraries that use JNI. Getting the power of Eclipse and Java would make making games in Panda3D 100 times more fun.

So, I’ve tried the Finder class and it seems to work perfectly
I’ll check if I can add some macros to the IDLE (the same as you added to the Emacs python-mode).

I only have 2 problems with the Finder.rebindClass function :

• it seems to erase the target python file
• with the carousel demo, it seems to reload new models instead of replacing them, so each time i run the function, all the pandas are cloned

I’ll see if I can understand these two points. More feedback soon !

Finder is probably writtten to assume that the input file was a temporary file, since it was written specifically to match the Emacs python-mode.el, which writes its input to a temporary file. Thus, it is appropriate for it to delete the input file when it has finished.

If your Python file does some work at the outer level, e.g. a call to loader.loadModel(‘panda.egg’), then that work will be done again every time you re-import the file.

This is one reason why it’s better to write your code so that everything is done within a function or a class.

David

To Appel,

You can call panda3D from a Java application already with Jepp
(Java embedded python).

I’have only tried it as a proof of concept but i was able to launch the samples from a java code …

But i remember i had to add a specific command line extension…

You are definitively right.
So, I commented out the concerned line in the Finder class. Then, i wrote 2 classes that allow dynamic rebinding triggered by file modification.

So, they monitor the source directory and reload a class file every time you save it. I like it this way because it does not depend on the editor you’re using.

I’m not an experienced python programmer, so feel free to comment

# FileChangeNotifier.py #

import os
import thread
import time

class FileChangeNotifier(object):

    def __init__(self, path, callback, interval=0.5):
        self.path_to_scan = path
        self.files = {}
        self.callback = callback
        self.interval = interval
        self.is_monitoring = False


    def __stat(self, filename):
        stat=os.stat(filename)
        return str(stat[6]) + '_' + str(stat[8])
    
        
    def __walkCallback(self, args, directory, files):
        for file_name in files:
            if file_name.endswith('.py'):
                filename = directory + '/' + file_name
                if self.files.has_key(filename):
                    nrepr = self.__stat(filename)
                    if not nrepr == self.files[filename]:
                        self.files[filename]=self.__stat(filename)
                        self.callback(filename)
                        
                else :
                    self.files[filename]=self.__stat(filename)
                

    def __monitor(self):
        while self.is_monitoring :
            os.path.walk(self.path_to_scan, self.__walkCallback, '')
            time.sleep(self.interval)

    def startMonitor(self):
        self.is_monitoring = True
        thread.start_new_thread(self.__monitor, ())

    def stopMonitor(self):
        self.is_monitoring = False

#########################

# ClassUpdater.py #

from direct.showbase import Finder
from FileChangeNotifier import FileChangeNotifier


class ClassUpdater(object):
    
    def __onFileChange(self, filename):
        try:
            Finder.rebindClass(None, filename)
        except Exception, ex:
            print 'Exception while rebinding the class :',ex
            
    
    def __init__(self, directory, interval):
        self.fcn = FileChangeNotifier(directory, self.__onFileChange, interval)
    
    def start(self):
        self.fcn.startMonitor()
    
    def stop(self):
        self.fcn.stopMonitor()


#########################

# Example of use (go.py) :

from ClassUpdater import ClassUpdater
from MyWorld import MyWorld


w = MyWorld()
c = ClassUpdater('.', 1)

c.start()
run()

I open a shell, start ppython, type “execfile(‘go.py’)” that execute the above script, and everything is fine.

Thanks a lot David, this is even better that what I was expected.
Is there a place where I could place this stuff in the wiki ? I also have a VC8 project compiling pview that could be useful, regarding to how many people ask how to use Panda in C++

(edit : fixed a “print” bug)

I’ve found this code to be very useful. Thank you very much for posting it.

All,

I have made these ‘dynamics’ work sometimes.
This is what I have done.

Files

ClassUpdater.py and FileChangeNotifier.py exist in the same
directory as the source programs World.py, MyWorld.py and go.py.

Of course, Finder.py has been modified in 3 places such that the the
immediately modified file does not get deleted.

All of this is described in the posts above.

The file OtherWorld.py would look like this.

import direct.directbase.DirectStart
from pandac.PandaModules import Lerp
from pandac.PandaModules import AmbientLight, DirectionalLight, LightAttrib
from pandac.PandaModules import NodePath
from pandac.PandaModules import Vec3, Vec4
from direct.interval.IntervalGlobal import *   #Needed to use Intervals
from direct.gui.DirectGui import *

#Importing math constants and functions
from math import pi, sin

class Mars:
  def __init__(self):
    return

  def setWorlddynamics(self, passedworld, mydynamics):
    var1 = 0.1
    var2 = 0.2
    var3 = var1 + var2
    if mydynamics == 'setamplitude':
      return var3
    elif mydynamics == 'setlerp':
      # 'AFTER execfile("go.py"), THEN THE FOLLOWING 
      # IS CALLED w = World(), THEN w exists.' 
      # 'AFTER run() THE OTHER METHODS ARE CALLED ONCE'
      # 'oscillatePanda IS CALLED LAST AND REPEATS'
      wo = passedworld
      #wo.resetsequence(wo.moves[1])  # WHEN UNCOMMENTED 
                                                         # DOES FREEZES THE PROGRAM
      for i in range(4):
        wo.moves[i] = LerpFunc(
                     wo.oscilatePanda,  #function to call
                     duration = 1000,   #3 second duration  
                                                #1000 IS BOUND BUT 
                                                #THE DURATION IS STILL 30
                     fromData = 0,  #starting value (in radians)
                     toData = 2*pi, #ending value (2pi radians = 360 degrees)
                                     #Additional information to pass to
                                     #self.oscialtePanda
                     extraArgs=[wo.models[i], pi*(i%2)]
                     )
      return
    else:
      return

The top (below the regular Panda import statements) of World.py would include this.

import OtherWorld

class World:
  def __init__(self):
    self.owm = OtherWorld.Mars()

    #This creates the on screen title that is in every tutorial
    self.title = OnscreenText(text="Panda3: Tutorial 2 - Carousel LLL",
                              style=1, fg=(1,1,1,1),
                              pos=(0.87,-0.95), scale = .07)

The bottom of World.py would look like this.

  def oscilatePanda(self, rad, panda, offset):
    # 'OSCILATEPANDA: AFTER A REBIND, 
    #THE CONTENTS OF THIS FUNCTION DOES NOT CHANGE.'
    #This is the oscillation function mentioned earlier. It takes in a degree
    #value, a NodePath to set the height on, and an offset. 
    #The offset is there
    #so that the different pandas can move opposite to each other.
    #The .2 is the amplitude, so the height of the panda will 
    #vary from -.2 to
    #.2
    # 'CHANGE 1.0 TO 0.0 OR 2.0. THEN REBIND. 
    # THE VALUE IS STILL 1.0 THIS VALUE DOES NOT CHANGE.'
    # 'TO CHANGE THE AMPLITUDE CALL THE FUNCTION setamplitude 
    # IN  THE CLASS Mars FROM THE MODULE OtherWorld.  
    # THE SCRIPT IS IN OtherWorld.py. ' 
    panda.setZ(sin(rad + offset) * 1.0 * self.owm.setWorlddynamics(self,'setamplitude'))
    self.owm.setWorlddynamics(self,'setlerp')


# w = World()
# run()

go.py would look like this.

from ClassUpdater import ClassUpdater
from World import World
from pandac.PandaModules import *


w = World()
c = ClassUpdater('.', 1)

c.start()
# run()

From the shell prompt, I would run ppython.bat
ppython.bat would look like this.

setlocal
set PATH=H:\Panda3D-1.3.2\python;%PATH%
H:
cd "H:\Panda3D-1.3.2\samples\Basic-Tutorials--Lesson-2-Carousel.0.4.0001"
ppython.exe -t
endlocal

From the ppython prompt, I would type and press return
after each of the following.

execfile("go.py")
run()

My my situation, my self-conclusion is the following.

The algorthm seems to ‘rebind’ World.py. This ‘really’ does not rebind.
The algorithm seems to ‘rebind’ MyWorld.py. The ‘really does’ rebind.

I can correctly rebind at ‘setamplitude.’ This works. I can just change the values of var1 and var.
These changes are immediately seen in the oscillation height of the Pandas.

I can not rebind at ‘setlerp.’ This does not work.
I have tried to add wo.moves[i].loop() in MyWorld.py. This makes the graphics become white.
I have tried wo.resetsequence(wo.moves[1]) This freezes the graphics. I can not get the graphics going again.

AIM

Great investigation.
Do you sometimes get the message : Warning: Finder could not find class ?
It seems to happen when rebinding imported modules front the World.py file… I don’t think there is a bug in the Finder.py class, we may don’t understand how to use it properly.

How is your Finder.findClass looks like ?
The original :

def findClass(className):
    """
    Look in sys.modules dictionary for a module that defines a class
    with this className.
    """
    for moduleName, module in sys.modules.items():
        # Some modules are None for some reason
        if module:
            # print "Searching in ", moduleName
            classObj = module.__dict__.get(className)
            # If this modules defines some object called classname and the
            # object is a class or type definition and that class's module
            # is the same as the module we are looking in, then we found
            # the matching class and a good module namespace to redefine
            # our class in.
            if (classObj and
                ((type(classObj) == types.ClassType) or
                 (type(classObj) == types.TypeType)) and
                (classObj.__module__ == moduleName)):
                return [classObj, module.__dict__]
    return None

My version :

def findClass(className):
    """
    Look in sys.modules dictionary for a module that defines a class
    with this className.
    """
    for moduleName, module in sys.modules.items():
        # Some modules are None for some reason
        if module:
            # print "Searching in ", moduleName
            classObj = module.__dict__.get(className)
            if (classObj and
               type(classObj) == types.ClassType and
               classObj.__module__ == moduleName):
               return [classObj, module.__dict__]
    return None

Finder.py is just the original …

H:\Panda3D-1.3.2\direct\src\showbase\Finder.py

with raytaller’s (3 times) commented out …

#os.remove(filename)

The os.remove(filename) is there of couse for emacs. Emacs uses temporary files.

I just used Notepad++ and regular ppython.exe to do my editing and running.

ynjh_jo, does your FindClass work better than the one in the distributed Finder.py? If so, how is it better?

AIM

At the first time before I really tried this dynamic approach, I made changes to Finder.rebindClass :

1. commenting out 3 “os.remove(filename)”
2. the original rebindClass only binds the 1st found class in a file. I want to bind every classes in a file, so I commented the “break” line and indent the lines below it until the line before the last “file.close()”.

About the findClass, if I still remember it correctly, the search process returned also if it found a type definition, which sometimes occur in sys.modules BEFORE the class itself.

I’m not sure, I’m not a professional programmer, why don’t you judge it ?
and I still have problem in finding the class instances.

BTW, this is my abandoned rebindClass (I dropped it since I don’t have enough time get this to work) :

def rebindClass(builtinGlobals, filename):
    file = open(filename, 'r')
    lines = file.readlines()
    for i in xrange(len(lines)):
        line = lines[i]
        if (line[0:6] == 'class '):
            # Chop off the "class " syntax and strip extra whitespace
            classHeader = line[6:].strip()
            # Look for a open paren if it does inherit
            parenLoc = classHeader.find('(')
            if parenLoc > 0:
                className = classHeader[]
            else:
                # Look for a colon if it does not inherit
                colonLoc = classHeader.find(':')
                if colonLoc > 0:
                    className = classHeader[]
                else:
                    print 'error: className not found'
                    # Remove that temp file
                    file.close()
                    # YNJH_JO : DO NOT REMOVE IT !!  IT'S NOT TEMP FILE !!
                    #os.remove(filename)
                    return
            print '==========================================='
            print 'Rebinding class name: [' + className + ']'
#            break

            # Try to find the original class with this class name
            res = findClass(className)

            if not res:
                print ('Warning: Finder could not find class')
                # Remove the temp file we made
                file.close()
                # YNJH_JO : DO NOT REMOVE IT !!  IT'S NOT TEMP FILE !!
                #os.remove(filename)
                return

            # Store the original real class
            realClass, realNameSpace = res

            # Now execute that class def in this namespace
            execfile(filename, realNameSpace)
            
            # That execfile should have created a new class obj in that namespace
            tmpClass = realNameSpace[className]

            # Copy the functions that we just redefined into the real class
            copyFuncs(tmpClass, realClass)

            # Now make sure the original class is in that namespace,
            # not our temp one from the execfile. This will help us preserve
            # class variables and other state on the original class.
            realNameSpace[className] = realClass

            # YNJH_JO : find and replace all instances of the old class
            '''
            print '/-----CLASS INSTANCES-----'
            main=sys.modules['__main__']
            myWorld=main.myRealTimeWorld
            myWorldDict=zip(myWorld.__dict__.values(), myWorld.__dict__.keys())
            for v, k in myWorldDict:
                # if it's a list or tuple, iterate through
                if type(v).__name__=='list' or type(v).__name__=='tuple':
                   print k
                   tmpSequence = []
                   for i in v:
                       if type(i)==types.InstanceType and str(i.__class__)==str(res[0]):
                          # class instance found !!
                          print '|  %s ( ID: %s ), name: %s' %(i, id(i), i.__class__.__name__)
                          print i.__dict__
                          tmpInstance = realClass()
                          tmpSequence.append(tmpInstance)
                       else:
                          tmpSequence.append(i)
                   myWorld.__dict__[k] = tmpSequence
                else:
                   if type(v)==types.InstanceType:
                      if str(v.__class__)==str(res[0]):
                         # class instance found !!
                         print '|  %s ( ID: %s ), name: %s' %(v, id(v), v.__class__.__name__)
      ##                   tmpInstance = realClass()
      ##                   i.__dict__ = tmpInstance.__dict__
                         #print i.__dict__

                print '|  --------------------------------------------'

            print '\-------------------------'
            '''

    # Remove the temp file we made
    file.close()
    # YNJH_JO : DO NOT REMOVE IT !!  IT'S NOT TEMP FILE !!
    #os.remove(filename)
    print ('    Finished rebind')

O.K. I wil try it out.

AIM

There should be a tutorial on this stuff. It makes Panda3D so much more interesting.

O.K.

I checked out your code. It looks pretty good. I am looking for time to implement it.

Here is an update.

I did manage to get World.py and Otherworld.py to both bind correctly using eclipse + pydev + pydev extensions. Before, just using ppython.exe, Otherworld.py would rebind. World.py would SEEM to rebind, but actually that file would not rebind.

I used the “remote debugging capability of the pydev extensions.”

I tested using ‘watered down’ simpler forms of World.py, Otherworld.py, and Finder.py with a different (but very similar) python executable.

go.py looks like …

from ClassUpdater import ClassUpdater
from World import World
import pydevd

w = World()
c = ClassUpdater('.', 1)
pydevd.settrace()
c.start()
while 1:
    w.manuallyorreoccuringprogramcall()

World.py looks like …

import OtherWorld

class World:
  def __init__(self):
    self.owm = OtherWorld.Mars()
    return

  def manuallyorreoccuringprogramcall(self):
    var = 0.0
    print var
    print str(self.owm.setWorlddynamics(self,'setamplitude'))
    print str(self.owm.setWorlddynamics(self,'setlerp'))

Otherworld.py looks like (the same as before) …

class Mars:
  def __init__(self):
    return

  def setWorlddynamics(self, passedworld, mydynamics):
    # USE LATER wo = passedworld
    if mydynamics == 'setamplitude':
      return 0.0
    elif mydynamics == 'setlerp':
      return 0.0
    else:
      return

Variable ‘var’ in anuallyorreoccuringprogramcall rebinds dynamically as I make changes to the progam file World.py on the fly.

AIM

How would you change a model dynamically?

I had the class:

class ball:
    def __init__(self):
        self.model = loader.loadModelCopy('smiley.egg')
        self.model.reparentTo(render)

When I created an instance, it renders the smiley model. I changed it too:

class ball:
    def __init__(self):
        self.model = loader.loadModelCopy('jack.egg')
        self.model.reparentTo(render)

I saved the file, dropped into the python prompt, used Finder.rebindClass and run() but it didn’t change! I dropped again and and made another one. It was jack instead of smiley, so the reload part apparently worked, but the old smiley remained.