Server-Client data transfers

Does anyone know a good way to transmit data between server and client scripts and keep it bundled somewhat like a dict? I know that only string variable and read only buffers can be sent between them, but I would like to have a functioning code that would do the same thing as…

Client

_rTransmit = { "Something":[ "an", "array", "of", "values", [ 1, 2, 3]]}
self.remote.send(_rTransmit)

Server

data = socket.recv(1024)
print data["Something"][0]
print data["Something"][1]
print data["Something"][4]

which would output…
an
array
[1,2,3]

Right now I’m trying to find a way to put this all in a single string and cleverly parse out the data on the recieving end, but I was wondering if anyone else had a better way.

Thanks.

Python uses “pickles”, never used them but the web full of examples.

Or you can send a singal string and then brake it up using self.yourstring.strip().split(" ") then call the word block you want, but that would take you knowing whats being sent in a way.

The “pickle” object actually works like a charm, thanks for the tip on that!

However, I’m having some errors in my server script…

It works fine for the first few transmitions from the client, then gives me an indexError…

EDIT**

Nevermind, it seems some of my transitions are somehow getting corrupt. They should all be in the same format as a dictionary reference, but it would appear not…

Are you using TCP or UDP?

TCP will help fixs some of the transitions error, cuase tcp makes the computer wait till all packets are send and working.

UDP doesn’t care and will just send the data even if some of it isn’t getting there.

server:

server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serversocket.bind(("192.168.1.103",4999))
server_socket.listen(10)

not sure if that tells you which one it is.

I added a try…except… statement when evaluating the pickle.loads() string and it seems that after the 4th or 5th transition from the client, half the pickles are corrupt and half are not, every other one.

Using pickle for networking would be the biggest memory leak ever.
With pickle, one can easily send malicious pickles that contain code that will be executed on the server when depickling.
From the Python website:

Personally, I use Panda’s modules.
panda3d.org/apiref.php?page=Datagram

ah, you answered another question I hadn’t asked yet. Thanks.

Does Panda’s modules only work in the panda networking or can it be used for any type?

In the case of Datagram, you can call getMessage and directly send that over a socket.

Do you have an example of it in use? adding a data type (e.g. a dict) to a data gram to send over a network then retrieving it as it’s original data type

EDIT** After playing around with it , I think I can adapt it to do what I want, just requires a bit more reworking of my transmitting data structure

I can’t seem to figure out how to compact it into a format that can be sent over a network.

I’ve packaged all my data bits into a Datagram with addString(), but now how to transmit them via a server link

small snippset from my networking code

def initNetwork(self):
      """
      sets up the basic networking stuff such as connection manager and such
      """
      self.cManager = QueuedConnectionManager()  
      self.cListener = QueuedConnectionListener(self.cManager, 0)
      self.cReader = QueuedConnectionReader(self.cManager, 0)
      self.cWriter = ConnectionWriter(self.cManager,0)
      self.connect()
      
      
  def connect(self):   
      """
      connects to the server
      """
      self.Connection = self.cManager.openTCPClientConnection("127.0.0.1",9099,6000)
      print "got connection"
      self.cReader.addConnection(self.Connection)
      taskMgr.add(self.readTask, "readTask", -39)

  
  def sendData(self,data):
    """
    sends the given data to the server
    """
    if self.connected == True:
      print data
      datagram = PyDatagram()
      datagram.addString(rencode.dumps(data,True))
      #print datagram
      self.cWriter.send(datagram, self.Connection)
    else:
      print "DATA NOT SEND. NO CONNECTION OR NOT CORRECTLY LOOGED IN ONTO SERVER"

the rencode function i’m using is simmilar to this one -> http://code.activestate.com/recipes/415503/ except that my version also supports unicode strings and zlib based compression of the data. its similar to pickle but ways saver since it uses struckts internally.

How does your readTask handle? I tried running my server.client.recv() through the taskMgr, but it crashes the client, causing it to freeze up, even when I’m not transmitting data from the server. I tried threading as well, but it seems to produce inconsistent streaming from the server, causing data to come in staggeringly.

Does the priority setting you added in make a difference?

i dont really remember the priorities but here’s the read-task

def readTask(self,task):
    """
    the task which is listening for incomming packages.
    depending on the package-type it redirects the contained data to different functions.
    """
    if self.cReader.dataAvailable():
      datagram = NetDatagram()
      #print datagram.getConnection()
      if self.cReader.getData(datagram):
        pkg = PyDatagramIterator(datagram)
        try:
          data = rencode.loads(pkg.getString())
        except:
          print "got corrupt data from the server.. what the heck.. and who hacked?.."
          data = {}