unicode text rendering with textNode?

I created a textNode and set a unicode font to it and passed a python unicode string to it, but the characters didn’t show up correctly.
I searched the manual and found this:

I added “text-encoding utf8” to Config.prc and the text shows up correctly now. But the rest of the info on the manual confuses me. Does this mean all strings I pass to panda must be in utf-8 now? And do the Python py files need to be in unicode if I read the strings from a text file?

It depends on which interfaces you are using to display your text. If you use the higher-level DirectLabel or OnscreenText, these will automatically process a python Unicode object correctly. If you use the lower-level TextNode interface, then you should call setWtext() instead of setText() when you’re passing a Unicode object instead of an ASCII string.

As to the manual page, that’s a bit out of date, but it’s still accurate enough for most of the other interfaces in Panda. It works because Python automatically converts Unicode objects to utf-8 when passing to Panda functions that accept a string. But this automatic behavior varies according to the Python installation.

David

So you’re saying that the config var will allow to not use setWtext() and still get the desired results, but it would require all the strings passed to panda to be unicode, and this might not work on some Python installations? But using setWtext() without any config var will allow me to use unicode strings were I want and might work with any python installation?

Approximately, yes.

To put it more precisely:

If you use setText(), you may pass either unicode objects or string objects. If you pass a string object, the string must either be strictly ASCII, or formatted according to the text-encoding config variable. If you pass a unicode object, then your Python’s “default encoding” must match the text-encoding config variable. The default encoding varies per installation, but it may be configured with sys.setdefaultencoding(), which must appear in a file named sitecustomize.py, per Python’s rules.

If you use setWtext() you may pass only unicode objects.

It is ideal, of course, if you pass a string object to setText(), and a unicode object to setWtext(). This is what the higher-level OnscreenText and DirectLabel objects do.

David

it seems like if I dont have the config var and pass a string containing unicode text to OnScreenImage, then it doesn’t render the text as expected. And I get this:

:text(warning): No definition in  for character U+xxxx

Also, this might not be a Panda issue, but I get an error if i pass a unicode text to OnScreenImage which ahd been processed with the module “textwrap”:

w = textwrap.TextWrapper(width = 10, break_long_words = True, replace_whitespace = False)
newtext = w.fill(mystring)
myonscreenimage.setText(newtext)

Error:

:express(warning): Non utf-8 byte in string: 0x93, string is '∩╝┤∩╜ê∩╜ë∩
╜ôπÇÇ∩╜ë∩╜
ôπÇÇ∩╜üπÇÇ
∩╜ô∩╜ê∩╜ë∩
╜å∩╜öπâ╝∩╜
è∩╜ë∩╜ôπÇÇ
∩╜ô∩╜ö∩╜Æ∩
╜ë∩╜Ä∩╜çπÇ
é'
Assertion failed: false at line 113 of c:\buildslave\dev_sdk_win32\build\panda3d
\panda\src\express\stringDecoder.cxx
:express(warning): Non utf-8 byte in string: 0x93, string is '∩╝┤∩╜ê∩╜ë∩
╜ôπÇÇ∩╜ë∩╜
ôπÇÇ∩╜üπÇÇ
∩╜ô∩╜ê∩╜ë∩
╜å∩╜öπâ╝∩╜
è∩╜ë∩╜ôπÇÇ
∩╜ô∩╜ö∩╜Æ∩
╜ë∩╜Ä∩╜çπÇ
é'
Assertion failed: false at line 113 of c:\buildslave\dev_sdk_win32\build\panda3d
\panda\src\express\stringDecoder.cxx
:express(warning): Non utf-8 byte in string: 0x86, string is '∩╝┤∩╜ê∩╜ë∩
╜ôπÇÇ∩╜ë∩╜
ôπÇÇ∩╜üπÇÇ
∩╜ô∩╜ê∩╜ë∩
╜å∩╜öπâ╝∩╜
è∩╜ë∩╜ôπÇÇ
∩╜ô∩╜ö∩╜Æ∩
╜ë∩╜Ä∩╜çπÇ
é'
Assertion failed: false at line 113 of c:\buildslave\dev_sdk_win32\build\panda3d
\panda\src\express\stringDecoder.cxx
:express(warning): Non utf-8 byte in string: 0x8a, string is '∩╝┤∩╜ê∩╜ë∩
╜ôπÇÇ∩╜ë∩╜
ôπÇÇ∩╜üπÇÇ
∩╜ô∩╜ê∩╜ë∩
╜å∩╜öπâ╝∩╜
è∩╜ë∩╜ôπÇÇ
∩╜ô∩╜ö∩╜Æ∩
╜ë∩╜Ä∩╜çπÇ
é'
Assertion failed: false at line 113 of c:\buildslave\dev_sdk_win32\build\panda3d
\panda\src\express\stringDecoder.cxx
:express(warning): Non utf-8 byte in string: 0x89, string is '∩╝┤∩╜ê∩╜ë∩
╜ôπÇÇ∩╜ë∩╜
ôπÇÇ∩╜üπÇÇ
∩╜ô∩╜ê∩╜ë∩
╜å∩╜öπâ╝∩╜
è∩╜ë∩╜ôπÇÇ
∩╜ô∩╜ö∩╜Æ∩
╜ë∩╜Ä∩╜çπÇ
é'
Assertion failed: false at line 113 of c:\buildslave\dev_sdk_win32\build\panda3d
\panda\src\express\stringDecoder.cxx
:express(warning): Non utf-8 byte in string: 0x82, string is '∩╝┤∩╜ê∩╜ë∩
╜ôπÇÇ∩╜ë∩╜
ôπÇÇ∩╜üπÇÇ
∩╜ô∩╜ê∩╜ë∩
╜å∩╜öπâ╝∩╜
è∩╜ë∩╜ôπÇÇ
∩╜ô∩╜ö∩╜Æ∩
╜ë∩╜Ä∩╜çπÇ
é'
Assertion failed: false at line 113 of c:\buildslave\dev_sdk_win32\build\panda3d
\panda\src\express\stringDecoder.cxx
:text(warning): No definition in  for character U+f2bd
:text(warning): No definition in  for character U+ffff
:text(warning): No definition in  for character U+ffff
:text(warning): No definition in  for character U+f2bd
:text(warning): No definition in  for character U+ffff
:text(warning): No definition in  for character U+ffff
:text(warning): No definition in  for character U+f2bd
:text(warning): No definition in  for character U+ffff
:text(warning): No definition in  for character U+ffff
Traceback (most recent call last):
  File "C:\Panda3D-1.8.0\direct\showbase\ShowBase.py", line 1708, in __igLoop
    self.graphicsEngine.renderFrame()
AssertionError: false at line 113 of c:\buildslave\dev_sdk_win32\build\panda3d\p
anda\src\express\stringDecoder.cxx
:task(error): Exception occurred in PythonTask igLoop
Traceback (most recent call last):
  File "main.py", line 10, in <module>
    run()
  File "C:\Panda3D-1.8.0\direct\showbase\ShowBase.py", line 2630, in run
    self.taskMgr.run()
  File "C:\Panda3D-1.8.0\direct\task\Task.py", line 502, in run
    self.step()
  File "C:\Panda3D-1.8.0\direct\task\Task.py", line 460, in step
    self.mgr.poll()
  File "C:\Panda3D-1.8.0\direct\showbase\ShowBase.py", line 1708, in __igLoop
    self.graphicsEngine.renderFrame()
AssertionError: false at line 113 of c:\buildslave\dev_sdk_win32\build\panda3d\p
anda\src\express\stringDecoder.cxx
Press any key to continue . . .

(this is when i do have the config var set. If I don’t, the text shows up, textwrapped, but of course the characters aren’t right.

:text(warning): No definition in  for character U+xxxx

This doesn’t sonud like an encoding issue (assuming U+xxxx is, in fact, the character you wish to render). It rather sounds like the character is missing from your font. Which font are you using? The egg file fonts provided with Panda only include ASCII letters, of course; you have to use a TTF or other dynamic font in order to render full Unicode characters. What characters are you trying to render, and are they reported correctly in the U+xxxx messages?

I don’t know anything about the module textwrap. Perhaps it is doing some re-encoding of the character strings in question. What kind of Python object does it return? Obviously, Panda’s auto-detection of Unicode objects can only work if you provide it a Python Unicode object.

David

it’s a pretty simple module:
docs.python.org/library/textwrap.html
This is different than Panda’s setWordWrap.

Oh, so I need to manually give OnScreentext unicode strings? OK, then:
textwrap seems to turn unicode strings to ascii, but i can reenocde it.

w = textwrap.TextWrapper(width = 10, break_long_words = True, replace_whitespace = False)
newtext = unicode(w.fill(args[0]))
	
myonscreentext.setText(newtext)

error:

line 55, in printStr
ing
    newtext = unicode(w.fill(args[0]))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 0: ordinal
not in range(128)

Seems like this problem is between you and textwrap. In my quick experiments, when fed a Unicode object, textwrap returns a Unicode object, but clearly that’s not the case for you for some reason. Are you sure you’re feeding a Unicode object in?

If you are feeding in a Unicode object to textwrap, and getting back a String object, then you’ll need to figure out precisely what textwrap is doing to make that conversion. Of course you can’t simply cast a String object to Unicode, without known what the encoding is. The way to convert an encoded String object is with the decode method, e.g. myUnicode = myString.decode(‘utf-8’)

David

You’re right. I seem to confuse encode() and decode()

Hey yo!

I want to put a text on the screen, as simple as that.

The problem it´s every manual I got about the subject, are not complete, I need a manual from the first instruction of de code font to the end.

Please don’t hijack an existing thread–this is a new question you are asking, that’s not really related to this thread.

As to your question, it’s not a very specific question. I think the manual is pretty clear about how to display text. The sample applications show plenty of examples, too. If you have already looked at these sources of information and need more help, you will have to ask your question a little more precisely.

David

Sorry, Your right!

I did it, cause that was the most similar forum I got!

But now I created this:

panda3d.org/forums/viewtopic … 4633#74633