Distributed Networking

[piratePanda]

I have been discussing the use of the distributed networking system with Drwr in private messages, and he suggested we move the conversation to the main forums so others may participate/benefit, so that’s what I’m doing here.

This is the conversation so far:

@piratePanda

Hello again, David.

As I think you know, I’m the guy who wrote the Beginner’s Guide to Panda3D 1.6.2 for Packt Publishing.

I’m working on a game that I’m hoping to get finished by the end of January to help alleviate my horrible under-employment. The game is intended to be multiplayer, over the internet. I want to use the distributed networking system for it.

To do that, I need to know how to use the distributed networking system. Since I’m going to learn the thing anyway, I also intend to write up some documentation for it to put in the panda3D manual, and I intend for that documentation to be as complete as reasonably possible. If that means writing another book, I’ll write another book. Hopefully it won’t be necessary to go to that extreme. I also want to create a sample program to be distributed with Panda3D.

I’m sending you this message because I know I’m going to need your help to learn the system.

I sent you a couple messages about it a while back, and you sent me some links to examples, but I had abandoned the project I was working on at the time without digging into the examples. The project was for my personal use, and I realized that the payout wasn’t worth the effort it would require.

I looked into the examples earlier this morning, and I ran into some problems that generated new questions.

First, I looked into a thread you linked me to where a gentleman named Zanz was doing the same exploration I am now. I copied down his first working example and tried to run it, but issues arose. For your convenience I’ve uploaded the files for his example and some panda3D files from the 1.7.2 build that I think are related to the issues to my web space and I’ll provide links to it, so you can look at the files in another browser tab while you read about the issues.

links omitted from quotation

Okay, so the first time I tried to connect to a server with his example, I got an error stating that the myClientRepository class had no attribute named sendSetZoneMsg. Such a method doesn’t exist in his file, so I looked in the ClientRepository.py his class inherits from, and I didn’t find that method, or any similar methods, there either. I also didn’t find it in the ClientRepositoryBase.py that ClientRepository inherits from and couldn’t find anything relevant. I didn’t dig any deeper down in the ancestry since it didn’t seem likely I’d find what I was looking for (a method that could be doing what sendSetZoneMsg is supposed to do).

I commented out the line in Zanz’s file that called myClientRepository.sendSetZoneMsg(self.zone) and tried again. This time I got an error telling me that myClientRepository had no attribute named createWithRequired. At this point I became fairly certain that the ClientRepository or ClientRepositoryBase class has under gone some significant changes since Zanz’s example was created. I looked in ClientRepository.py again and I found a couple of methods that seem to be likely renames/rewrites/substitutes for the createWithRequired method, those being generateWithRequiredFields, generateWithRequiredOtherFields,
and generateWithRequiredOtherFieldsOwner. None of those take the correct number of arguments though.

That’s as far as I got with Zanz’s example.

I also tried your Tagger game. I downloaded the source using the link from the end of your forum post and ran two copies of the game. The first one I gave the option -s to start a server, the second I gave the option -p 192.168.1.65:4430 to try and connect to the server I was running on the same computer. I got the IP address from my OS and the port from the print out of the tagger game, which stated that it couldn’t connect to server at 127.0.0.0 port 4430. When I did this, the client program loaded up and I entered my street name, then it went to a gray screen that said “waiting for server” and sat there for 5 minutes or so before I closed it. I have no real idea what’s going on there.

I realize I’m getting long in the tooth here, and I apologize for that. I’d appreciate it if you could either help me get these examples working, or better yet, if you would be willing to help me get the distributed system working for my game. I’d happy to share my game’s source code at your request, if you’re up to helping on that.

@drwr

Hi,

Hmm, I didn’t realize that Zanz’s code was so stale. That’s unfortunate.

Tagger should run well, though. You might need 1.8.0 instead of 1.7.2, but I think it will run OK on 1.7.2 as well.

The thing about Tagger is that you need three components: there is the server, which manages communications back-and-forth, and there is the AI, which connects to the server and creates the mazes, and there is the client, which connects to the server and plays the game. Without the AI, the client will just wait and wait and wait.

Since the default behavior is to contact the localhost server, you don’t actually need to specify your own IP address anywhere. Just open three windows, and run the following three commands in them, one per window:

python main.py -s
python main.py -a
python main.py -c

You can actually combine the server and the AI into the same process, so you really only need two windows:

python main.py -s -a
python main.py -c

You can almost run -s -a -c to run server, AI, and client all in the same process, but this doesn’t work on 1.7.2. It will work on the most recent commits I’ve made to 1.8.0, but that’s probably not important to you to help understand what’s going on. Smile

David

[piratePanda]

@piratePanda

I’ve been looking through your code for Tagger, focusing on the setup of the Server and Client repositories.

What is the purpose of the threadedNet variable? Does it put the networking into a separate thread so it runs asynchronously? I’m not up to speed on exactly what threading is or how it works.

There a couple times when you use methods of base.config.

tcpPort = base.config.GetInt('server-port', Globals.ServerPort)
hostname = base.config.GetString('server-host', Globals.ServerHost)

I printed out base.config to find out that it was a DConfig object and looked up the DConfig class in the API, and I genuinely can’t figure out what those methods you’re calling do. It looks to me like you’re turning an Int into an Int and a String into a String. I don’t get what you’re doing with that, could you explain it?

My last question is related to some basic code I put together to try and get a server/client connection started. I made two files, one for the server and one for the client. I ran the server first, and it ran fine. I ran the client next and I got an error during my importing of ClientRepository, which doesn’t make much sense to me because I can still run Tagger fine and it uses the same import statement. The error refers to line 11 of DistributedOjbectBase, which reads:

notify = directNotify.newCategory("DistributedObjectBase")

It’s a NameError, and says the name directNotify is not defined. The traceback tells me that DistributedObjectBase is imported by DistributedObject, which is imported by CRCache, which is imported by ClientRepositoryBase, which is imported by ClientRepository. Why would I get this error when I import ClientRepository in my file, but not when TagClientRepository.py imports it?

@drwr

Hmm, maybe we should put this thread in the main forum, so others can benefit from (or contribute to) this discussion as well? Smile

@piratePanda

What is the purpose of the threadedNet variable? Does it put the networking into a separate thread so it runs asynchronously? I’m not up to speed on exactly what threading is or how it works.

If true, this tells the underlying networking system to use threads to listen for incoming data. It has a subtle effect on performance, but can also occasionally cause problems; this is why it’s an option–mainly for experimentation.

@piratePanda

There a couple times when you use methods of base.config.

tcpPort = base.config.GetInt('server-port', Globals.ServerPort)
hostname = base.config.GetString('server-host', Globals.ServerHost)

This is an older, now deprecated interface to reading the .prc file. I should have used the newer interface instead, which looks like this:

tcpPort = ConfigVariableInt('server-port', Globals.ServerPort).getValue()
hostname = ConfigVariableString('server-host', Globals.ServerHost).getValue()

@piratePanda

It’s a NameError, and says the name directNotify is not defined.

directNotify is one of many symbols placed in the globals by ShowBase, so if you haven’t created a ShowBase instance yet it doesn’t exist unless you import it explicitly. You can import it explicitly from direct.directnotify.DirectNotifyGlobal if you need to access it before ShowBase.

David

[piratePanda]

@piratePanda

I don’t mind moving this discussion to a forum thread if that’s what you’d like to do.

About the two lines related to the config file, are they reading those two variables and overwriting the values in Globals? If so, why are you getting values from the config file for this?

I’ve succeeded in connecting a client to a server with the following code:

server.py:

from direct.distributed.ServerRepository import ServerRepository
import direct.directbase.DirectStart

class TestServerRepository(ServerRepository):
    def __init__(self):
        tcpPort = 4430
        hostname = ''
        dcFileNames = ['direct.dc']
       
        ServerRepository.__init__(self, tcpPort, serverAddress = hostname,
                                  dcFileNames = dcFileNames)

        # Need at least 32 bits to send big picture packets.
        self.setTcpHeaderSize(4)

        # Allow some time for other processes.
        base.setSleep(0.01)
      
TSR = TestServerRepository()
run()

client.py:

import direct.directbase.DirectStart
from direct.distributed.ClientRepository import ClientRepository
from pandac.PandaModules import *

class TestClientRepository(ClientRepository):
   def __init__(self):
      dcFileNames = ['direct.dc']
      
      ClientRepository.__init__(self, dcFileNames = dcFileNames,
                          connectMethod = self.CM_NET)
      
      # Need at least 32 bits to receive big picture packets.
      self.setTcpHeaderSize(4)

      # Allow some time for other processes.  This also allows time
      # each frame for the network thread to run.
      base.setSleep(0.01)
      
      self.url = URLSpec('g://%s:%s' % ('localhost', 4430))
      
      self.connect([self.url],
                successCallback = self.connectSuccessMsg,
                failureCallback = self.connectFailureMsg)
      
   def connectSuccessMsg(self):
      print("Successfully Connected")
      
   def connectFailureMsg(self, statusCode, statusString):
      print("Failed to Connect")
      print(statusCode)
      print(statusString)
      
TCR = TestClientRepository()
run()

But I’m not quite sure what the next step is. Could you give me a hint?

@drwr

I read these values out of the config file to give the developer the option of setting them there. For instance, I used this feature when I was developing this code across several different environments. When I’m working on my laptop, I might need to use a different hostname:port than when I’m working on my desktop. By putting this information in the config file, I can set it differently on each computer, while keeping the code the same, so I don’t have to keep hand-editing the code every time I switch from one computer to the other.

In your project, the next thing to do is to create some distributed objects.

Hmm, have you seen my post on page 3 of the zanz thread where I post a link to some sample code that demonstrates some of the fundamentals of DistributedObject creation?

David

[piratePanda]

@piratePanda

I finally dragged myself away from the new computer games that came with Christmas to get back to work on figuring out the distributed object system. I found those examples you mentioned and I’ve been looking through them.

In client1.py I found something interesting I’ve never seen before, and I’d like to ask you about.

Several methods that I would expect to be called on a NodePath, like setColor, you are calling on the custom class instance itself. Furthermore, there aren’t any custom definitions of these methods in the class definition. I think you’re able to do this because of the following call I found in the DistributedAvatar class’s init method:

NodePath.__init__(self, 'avatar')

I’ve never seen NodePath used like this before. Could you explain to me what’s happening with this call?

@drwr

It’s simply that DistributedAvatar inherits (eventually) from NodePath, the same way that Actor inherits from NodePath.

Calling Parent.init(self) from the init() function of the subclass is just the normal Python way to upcall to your parent class’s constructor.

[piratePanda]

@piratePanda

DistributedAvatar inherits from NodePath because it inherits from DistributedNode, then?

@drwr

Right.

[piratePanda]

@piratePanda

Just to give you a status update, I’ve moved out of the testing program stage and started implementing distributed networking in the game I’m working on.

I have a couple more questions for you, if you don’t mind.

Firstly, when should I use DistributedObject and when should I use DistributedNode? I looked through your example app Net and I couldn’t see a clear cut difference between the things you made DistributedObjects and the things you made DistributedNodes.

In the custom .dc file for Net, net.dc, I noticed that the methods tended to have a division by some number following their arguments. It looks to me like this is a division used to convert floats to ints for ease of network transmission, but that kind of operation would require a multiplication somewhere to balance the division, and I can’t find that multiplication. Am I just missing it somewhere, or am I missing or misunderstanding something on a grander scale?

Is there a call to bring a new client fully up to speed with all DistributedNodes/DistributedObjects, or does this happen as objects send updates? I ask because I have my program creating a serverRepository, connecting to it with a clientRepository in the same program, and then creating a DistributedNode with that clientRepostitory. This works fine, but when I connect with another copy of the program, one which isn’t creating a serverRepository, I get a connection success call back but the DistributedNode doesn’t show up.

Do methods identified in the .dc file every automatically broadcast, or is a manual update always required?

[drwr]

Firstly, when should I use DistributedObject and when should I use DistributedNode? I looked through your example app Net and I couldn’t see a clear cut difference between the things you made DistributedObjects and the things you made DistributedNodes.

A DistributedNode is just a class that inherits from DistributedObject and PandaNode (actually, NodePath). So it is both a distributed object and it is a node. You should use DistributedNode whenever you have something that wants to be inserted into the graph, like a node–something that has position, rotation, maybe children, and so on, and will presumably contain renderable geometry–and use DistributedObject whenever you have something that does not have these properties.

In the custom .dc file for Net, net.dc, I noticed that the methods tended to have a division by some number following their arguments.

This is just a syntactic convention used by the dc file. It doesn’t actually mean only division; it actually means that Panda will multiply your parameters by this factor when sending messages, and dividing by the same factor when receiving them. It’s just a trick to send a decimal number via an integer. For instance, DistributedNode.setX() sends its parameter as int16 / 10, which means the legal values for X are in the range -3276.8 to 3276.7. (That is, you can store -32768 to 32767 in an int16, and the Panda convention allows you to store up to one decimal point of precision.) An int16 is half the size of a float32, so it’s a win if every byte counts. Mostly, nowadays, every byte doesn’t count for much, so there’s not a whole lot of reason to use this convention often.

Is there a call to bring a new client fully up to speed with all DistributedNodes/DistributedObjects, or does this happen as objects send updates?

This is what the keywords following the message definitions are for. If the keyword “ram” is present, it means that the message is persistent, and the server will automatically retransmit it to any new clients who see this object. (It does this by calling the corresponding getFoo() method when needed.) If the keyword “ram” isn’t present, then the message is not persistent, and new clients who see the object won’t hear the message. The idea is that “ram” should be added to messages that are part of the object’s state, like its color, position, whatever, and it should not be added to messages that are transitory in nature (like a command to play a sound effect or something).

Do methods identified in the .dc file every automatically broadcast, or is a manual update always required?

A manual update is required. Our convention is to write a trio of Python methods for each Distributed method: foo(), which does the operation locally, d_foo(), or “distributed foo”, which sends the message to the distributed system, and b_foo(), or “both foo”, which calls d_foo() and foo() together.

David

[piratePanda]

Is there a way to tell a clientRepository to generate DistributedObjects that exist in a zone other than having it create a DistributedObject in that zone?

I ask because I want clients that connect to a host to create only one distributed object, and that object is dependent on other distributed objects.

[drwr]

Yes; that’s the set of “interest zones”. The client maintains a set of zoneIds that it is interested in hearing about. By default, this set is the same set of zoneIds that it has created DistributedObjects in; but it can also include any additional zones you specify.

Use ClientRepository.setInterestZones() to add additional interest zones.

David

[piratePanda]

Excellent, thank you.

[piratePanda]

When a client connects to a server in this system, does the client get some kind of unique identifier to differentiate it from other clients? Or is there a way for a client to get a count of how many other clients are connected to the same server?

I’m looking for a way for clients to uniquely identify objects they create. For instance, the first client my preface object names or id numbers with a “A”, while the second client uses “B”, and so on.

My reason is that my clients are each creating DistributedNodes at regular intervals, and I need to be able to tell those DistributedNodes apart on all clients because other DistributedNodes need to be assigned to them. The plan is tell children “You’re a child of A1” or some such, so the child can find the correct parent.

[drwr]

Each client is assigned a range of legal doId’s, so you can tell which object belongs to which client by examining its doId.

In particular, you can call ClientRepository.isLocalId(doId), which returns True if the doId belongs to this client, or False if it belongs to some other client. There’s also ServerRepository.getDoIdBase(doId), which returns the client’s doIdBase given the object’s doId. This is only a server-level call, though.

There’s no easy way for a client to know how many other clients are connected to the server, but you could build this into your particular game by having all clients create an “I am here” object in a common zone, or something like this. (Of course, this requires that you trust all of your clients, which is generally a bad idea in a distributed environment. But much of the ClientRepository/ServerRepository infrastructure suffers from this same trust issue anyway; and that’s an issue for another discussion.)

David

[piratePanda]

So, what you’re telling me is that I need to learn C++ and write my own Distributed Networking system for scratch, eh?

Kidding, of course.

Thanks for the info. That “I am here” object sounds like a good idea. I can probably build that into the player object I’m using to store their information, like color and team.

[piratePanda]

Is there a way to kick a client from a server?

[drwr]

No, only the clients can decide when to connect or disconnect, but you can build a “please disconnect” message, which you can program your clients to respect.

If you don’t trust your clients, of course, we have bigger issues anyway, as I already mentioned.

David

[piratePanda]

It’s more for a host wanting to kick another player out of the waiting room prior to starting a match.

[piratePanda]

How would I build such a message?

Also, I did find a way to kick a client while you waiting for your last post. It was just… unpleasant.

I looked up all the clients with the clientsByDoIdBase, got a Client object from the list, got it’s connection, got the connection’s manager, and told the manager to close the connection.

Not a very happy way to go about it, certainly.

[drwr]

How would I build such a message?

The same way you build any message–with a DistributedObject method that, when the server calls it, the client disconnects.

Also, I did find a way to kick a client

Ah, good point. That’s actually a fine way to do it, though it is messy for both the server and the client. For the purposes of making a good show for a well-behaved client, it makes more sense to use a message that says “leave the waiting room”.

David

[piratePanda]

Is there any way to make a particular client ignore a class of DistributedObjects, or DistributedObjects with a certain zoneId? I ask because I’m getting errors when my AIRepository can’t create an AI version of a DistributedObject, regardless of it’s zoneId, and I dislike having to create an AI version of an object the AI doesn’t need to touch, and never will.

[drwr]

I don’t think there’s any way built in to handle that currently. The assumption is that the AI will want to meddle with (or at least know about) all of the objects in the world.

David