DistributedObject error

[Ogel]

We’ve bumped into a pretty strange error, and I was hoping someone might be able to help out. We have a networked game where a number of clients participate in a series of actions together. We’re using a few finite state machines (for each client avatar and one for the server) to facilitate these actions.

When an avatar left the game, the code wasn’t properly “cleaning up” his/her action state on the server. So, we placed a method call in the avatar’s “delete” function that tells it to vacate the server’s FSM. However, this causes an unexpected problem:

Traceback (most recent call last):
  File "server.py", line 133, in ?
    run() # run Panda
  File "C:\Panda3D-1.0.5\direct\src\showbase\ShowBase.py", line 1603, in run
    self.taskMgr.run()
  File "C:\Panda3D-1.0.5\direct\src\task\Task.py", line 781, in run
    self.step()
  File "C:\Panda3D-1.0.5\direct\src\task\Task.py", line 728, in step
    self.__stepThroughList(taskPriList)
  File "C:\Panda3D-1.0.5\direct\src\task\Task.py", line 671, in __stepThroughList
    ret = self.__executeTask(task)
  File "C:\Panda3D-1.0.5\direct\src\task\Task.py", line 602, in __executeTask
    ret = task(task)
  File "C:\Panda3D-1.0.5\direct\src\distributed\ServerRepository.py", line 188, in readerPollUntilEmpty
    while self.readerPollOnce():
  File "C:\Panda3D-1.0.5\direct\src\distributed\ServerRepository.py", line 201, in readerPollOnce
    self.handleDatagram(datagram)
  File "C:\Panda3D-1.0.5\direct\src\distributed\ServerRepository.py", line 221, in handleDatagram
    self.handleClientUpdateField(datagram, dgi)
  File "C:\Panda3D-1.0.5\direct\src\distributed\ServerRepository.py", line 266, in handleClientUpdateField
    dclass = self.DOIDtoDClass[doid]
KeyError: 2000001

This error seems to occur at the end of the delete function at this line:

DistributedObject.delete(self)

We tried placing our method call in disable(), but the same problem occurs (when it hits “DistributedObject.disable(self)”).

We don’t see this error when we remove our clean-up method call, so our attempts to remove this avatar from the server FSM is definitely doing something wrong. I’m not very familiar with DistributedObject methods, so I’m not sure why disable() and delete() are coughing. Does anyone have any ideas as to what causes this error?

[drwr]

delete() and disable() are not meant to be called directly. They are called internally when the appropriate message is received from the server (or when the DistributedObject is removed from the cache).

If you need a cleanup function, make a new function, for instance cleanup(), and call it from delete, and/or call it directly on program exit.

David

[Ogel]

Thanks for your help. I’m sorry if my previous post was unclear, but we’ve redefined disable() and delete(). We’re doing some clean up in those functions and then calling “DistributedObject.delete/disable()” at the end. However, we’re not calling those functions directly - we’re letting the system do it when a client exits the program. The error message is appearing on the server (which crashes, obviously ).

Should we not redefine disable() and delete()? Does this error mean that somehow the DOID is already gone by the time “DistributedObject.delete()” is called?

I’ve been trying to think of what our clean-up method might be doing to cause this error. Unfortunately, it’s just telling a server-side FSM to move one of its states from “occupied” to “unoccupied”. If you have any other ideas please let me know - thanks!

[drwr]

Oh, OK. Yes, in principle, it should be fine to put your cleanup code in disable() or delete(). You should be aware that both functions will be called when the object is deleted, so if you put the cleanup code in both functions, it will get called twice.

I suspect something is happening as a side-effect of switching the FSM that is causing problems. The particular crash you are getting indicates that someone has attempted to send an update on a deleted object–I bet one of your cleanup methods is (indirectly?) trying to send an update of some kind on this object. Too late for updates!

Try putting print

[Manakel]

Hello ,

Totally out of topic, but how far did you manage to go with the
Distributed Object part of the PandaEngine.

I though it was not working except for things like ChatRoom?

Maybe i should rewrite my code to use Panda Distributed Object system and
not the old Dopy one…

[russ]

Distributed Objects work in panda. The code from the chat room can be extended into larger programs. There are some workarounds at startup that need to be accounted for but otherwise a server and several clients can communicate without any problems.

What is meant by “the old Dopy one”?

[Manakel]

Actually i use Dopy as distributed object service.
It’s very crude, no more maintained and does not support advanced features
(like only way to stop the connetion is to kill the app).
Howewer any object that can pickle is directly available & synchronized in the network with one line of code… (no description file, no mark up language)…

[bigfoot29]

Sorry for bumping that topic…

After I messed around with twisted a bit I at least accepted for myself that I am far too stupid to get this working. Sure, I can start a chat server, but I can’t manage to get a client working that goes to send a user defined (means a user typed) message to the server except those messages that are integrated into the protocol - so for now away with twisted.

But after looking into the distributed code a bit I found out that this doesn’t seem to be easier. Is there a module out there that allows SIMPLE network access? I have no clue why I need to have distributed objects just for the reason to have a chatserver/client combination…
Twisted might be useable for a server, but where can an easy to manage client package be found? I haven’t tried dopy because, well, the code should be able to clean itself if needed without shutting down the application.

However… I have tried the Networking tut with the chat system where I stumbled upon the fact that I can’t even determine if the program wants to be server or client. Why? Because it doesn’t accept the “/” as a symbol.
Does the Tut still have the problem that you can’t choose a long nickname and then having trouble with the text messages?

Or can you give me a hint why the distributed object system is so much better than the “raw” networking code? I don’t see any advantages for games here… (well, I am clueless about this)
As far as my understanding goes: This is helpful if you want to distribute work (means threads) to other machines (-> cluster work) and thus recieve the results from the distributed object. But for a server/client usage?

Questions, Questions

Regards, Bigfoot29

[drwr]

The DistributedObject system is meant to be a higher-level interface than “raw” networking. As such, it handles some things for you automatically, like packing up your Python objects into a byte stream for sending over the network, and then decoding them to the same Python objects on the other side, and calling the same named Python function on the other side.

The interface presented by the DistributedObject system allows you to think of having just on Python object that exists on multiple different machines, and when you change a property on one instance, it automatically changes on all of the other instances too. That’s pretty handy.

However, there is still a certain amount of stuff you have to understand and know about in order to use DistributedObjects properly.

If all you want is to send a byte stream from one client to another, DistributedObject (as well as Twisted) may be more than you need. You can use Python’s socket library to open raw TCP/IP connections between clients. (But again, you have to know something about what you’re doing.)

I think it all comes down to documentation. Some networking systems handle more stuff automatically than others, and the stuff that any given system handles for you may or may not mesh well with your needs. But with any networking system, you have to know something about how it is intended to be used.

Presumably, when the DistributedObject system is better documented, it will be easier to use.

David

[bigfoot29]

You are right Yellow and I are doing some networking stuff independend from each other (ok, my work is based on his posted client, but we are talking it over so some of his implementations might have some of my ideas… maybe )

The distributed object might be handy as you said, but as long as I am staring onto a small text telling me to ask in the forums I am somewhat helpless… looking into sourcecode (like Yellow did with the cluster code to create the server/client thingie) isn’t my way as long as I have NO basics I can rely on (a working basic server I mean)

However, its nice to have you back.

Comes time, comes knowledge… an/or someone who will do a proper networking manual section… because thats what most of the free 3D-Engines are lacking too… might be a big plus to have that properly documented in a noob proof way

Regards, Bigfoot29

[Marc]

Since you’re up to learning it so well YOU could be the one to write this manual section, what do you think?

[Ogel]

Hello, again. Sorry to bump this thread, but I have a related question.

We’re creating a number of distributed objects (an avatar for each player) using “createWithRequired”. Now we’d like to restart the game (without shutting down every client) and allow players to select a new avatar. To be safe, we’d like to delete the avatar objects and create fresh ones.

How can we delete distributed objects without having the client exit? David’s earlier post said that we shouldn’t call delete() or disable() directly. Is there another way for the server to kill an object?

-Ogel

[Yellow]

first make sure all references to the node are removed, then call:

nodepath.removeNode()

this efficiently deletes the node.

[drwr]

To delete a DistributedObject, you need to ask the server to delete it. I’m not actually that conversant with the ServerRepository code; in the VR Studio, we use a completely different (proprietary) server to handle our distributed transactions. But from examining the code in ServerRepository and ClientRepository, it looks like the idea is to call distributedObject.sendDeleteMsg(), which should delete all of the objects on all of the other clients and on the server. It’s not clear how you’re supposed to then delete the local copy. Maybe clientRepository.deleteObject(obj)? Did this method get lost in the transition to 1.1?

David