PTA_double

memecss · February 24, 2010, 7:03pm

Hi,
I am getting this error:

../pgraph/Opt3-Linux/libpgraph.so: undefined reference to `Dtool_PointerToArray_double'

while using PTA(double) in pgraph/shaderInput.
Any possible solution?

thanks

drwr · February 24, 2010, 7:05pm

We haven’t exposed PTA(double) to interrogate yet. The easiest solution is to use PTA(float) instead. If you really want PTA(double), you should add it to the list of things that are exposed in panda/src/express/config_express.N.

David

memecss · February 24, 2010, 7:20pm

mmh, I have added these four lines in panda/src/express/config_express.N,

forcetype PointerToArray<double>
forcetype ConstPointerToArray<double>
renametype PointerToArray<double> PTADouble
renametype ConstPointerToArray<double> CPTADouble

but still I am getting the same error. Should I recompile the dtool directory?

thanks

drwr · February 24, 2010, 7:59pm

It might also be necessary to define a pta_double.h/.cxx file, and add it to the build scripts, in order to force the template to be instantiated.

It shouldn’t be necessary to rebuild dtool, but it might be necessary to make clean in panda/src/express, since changing the .N file might not be enough to force interrogate to re-run in that directory. Are you using makepanda or ppremake?

David

memecss · February 24, 2010, 8:12pm

I am using ppremake. Cleaning and recompiling both dtool and panda solved the problem.
It should work the same also for LVector and LMatrix right? and how do I add pta_double.h it to the build script?

drwr · February 24, 2010, 8:18pm

Add it to the Sources.pp file, wherever you see PTA_float referenced. But if it worked without doing that, then I wouldn’t worry about it.

Right, PTA(LVecBase3f) and PTA(LMatrix4f) should work the same way. Note the conceptual difference between LVector3f, LPoint3f, and LVecBase3f: the first two have specific semantics, and the third is just a generic 3-component value, with no particular semantics. Lots of people are confused by this and use LVector3f wherever they really mean LVecBase3f.

David

memecss · February 24, 2010, 8:42pm

I was one of them. So does a PTA(BaseClass) behave polymorphically?

memecss · February 24, 2010, 8:50pm

By the way pta_double.h is already defined in panda/src/putil

drwr · February 24, 2010, 10:30pm

No more nor less than BaseClass does itself. Typically you need an array of pointers if you want polymorphic behavior, so you’d need PTA(BaseClass *) or PTA(PT(BaseClass)) if you really want an array of polymorphic types. For that kind of thing, you should probably go with something explicit like a TextureCollection rather than using PTA.

But, polymorphism isn’t required for LVecBase3f and related classes. Each of these is usually used as a concrete, and each of these can be cast as needed to any of the other types–like an int to float. If you create an PTA(LVecBase3f), it can be used to store LVector3f’s; they’ll just be implicitly cast to LVecBase3f when they’re stored.

Ah, probably best to move the interrogate lines from config_express.N to config_util.N, then.

David

memecss · February 25, 2010, 8:23pm

mmh, I moved the lines in to config_util.N, pta_double.h/cxx were already in putil/Sources.pp and now I get this error:

../putil/Opt3-Linux/libputil.so: undefined reference to `Dtool_PointerToBase_ReferenceCountedVector_double'
../putil/Opt3-Linux/libputil.so: undefined reference to `Dtool_PointerToArrayBase_double'

I am still missing something right?
And what about exposing PTA(LMatrix) and PTA(LVecBase) should I do that in src/linmath/?

One more question:
Right now I have a new function setShaderInput() that takes PTA(float/double) and send the data to the GPU nicely passing PTA().v() to cgSetParam/Array/Matrix… Now I want to overload it with PTA(Lmatrix/LVecBase) but I am worried about the number of conversions that data will go through: python array of LMatrix -> PTA(LMatrix) -> PTA(float). Do you think is there a better way?

drwr · February 25, 2010, 8:36pm

Ahh, I’ve given you bad advice, my apologies. I remember this problem, it happened to me too. I should have known better.

The problem is that the template class PointerToArray is defined in express, so interrogate wants to assume (incorrectly) that any instantiation of that template is also defined in express. The upshot is that you can only publish PTA(foo) vectors from express, not from any other directory. So, you’ll need to move those lines back into config_express.N.

For the same reason, you’ll have to add the PTA(LMatrix/LVecBase) to the same place. No need to create a PTA_LMatrix or PTA_LVecBase class for them, though; we mainly did this for PTA_float and PTA_double and such to support MSVC6, which we are no longer worried about supporting.

David

drwr · February 25, 2010, 8:39pm

Oops, forgot to answer this point.

I don’t know if you need that last conversion step. Can’t you just pass the PTA(LMatrix).p() to the shader? Since an LMatrix4f is stored as sixteen consecutive floats, that can be treated the same as an array of (16 * n) floats in C++.

David

memecss · February 25, 2010, 9:33pm

Thanks a lot that worked.

No Cg only takes pointers to 1D arrays. I could send to the GPU n times arrays of (16 * 1) floats instead. I assume it’s going to be slower right? But probably better then copying the PTA(LMatrix) into a PTA(float).

memecss · February 25, 2010, 10:04pm

Sorry to bother you again with this but adding

forcetype PointerToArray<LVecBase3f>
forcetype ConstPointerToArray<LVecBase3f>
renametype PointerToArray<LVecBase3f> PTAVec3
renametype ConstPointerToArray<LVecBase3f> CPTAVec3

doesn’t seem to be enough. I am getting:

Type unknown is unknown to parser.
Invalid CPPSimpleType: 7
Type unknown const & is unknown to parser.
Type unknown is unknown to parser.

...

Type unknown const & is unknown to parser.
g++ -ftemplate-depth-30  -c -o Opt3-Linux/libexpress_igate.o -I. -I/home/memecs/Perforce/perforce.etc.cmu.edu_1666/p4v-local/panda3d-1.7.0.1/panda -I../pandabase -I/usr/local/panda3d/include -I/usr/include/python2.6    -g -O2 -fPIC Opt3-Linux/libexpress_igate.cxx
Opt3-Linux/libexpress_igate.cxx:260: error: ‘unknown’ was not declared in this scope
Opt3-Linux/libexpress_igate.cxx:260: error: template argument 1 is invalid
Opt3-Linux/libexpress_igate.cxx:260: error: invalid type in declaration before ‘;’ token
Opt3-Linux/libexpress_igate.cxx:265: error: ‘unknown’ was not declared in this scope
Opt3-Linux/libexpress_igate.cxx:265: error: template argument 1 is invalid
Opt3-Linux/libexpress_igate.cxx:265: error: invalid type in declaration before ‘;’ token
Opt3-Linux/libexpress_igate.cxx:270: error: ‘unknown’ was not declared in this scope
Opt3-Linux/libexpress_igate.cxx:270: error: template argument 1 is invalid
...

Should I add header and source to Sources.pp in src/express

drwr · February 25, 2010, 10:08pm

Interrogate hasn’t read the linmath sources: nothing in express includes them, so it has no idea what an LMatrix4f is.

Yikes, trouble is, linmath depends on express, not the other way around; so it’s impossible for code in express to include a header file defined in linmath. This, combined with the bug in interrogate, makes it impossible to publish a PT(LMatrix4f) via interrogate.

Maybe it will be possible to fix this interrogate bug. Let me investigate.

David

memecss · February 25, 2010, 10:13pm

Ok thanks a lot.

drwr · February 25, 2010, 10:28pm

Oops, I missed this reply earlier.

I think there’s a confusion. When I said “an array of (16 * n) floats”, I meant a 1-d array of floats, of size 16 * n, where n is the number of matrices.

That is:

PTA(LMatrix4f) myarr = PTA(LMatrix4f)::empty_array(4);

defines an array of four matrices. You can cast this:

const float *f = (const float *)myarr.p()

which becomes a linear array of 64 floats, and pass this to the CPU via the appropriate Cg call.

Note that making multiple calls into Cg should be avoided whenever possible. This is almost certainly slower than doing any recopying needed and then making a single call into Cg.

David

memecss · February 26, 2010, 12:07am

mmh, I am not sure how to proceed at this point. I’d like to stick with PTA(float/double) to keep the code more clean and generic. What if the user convert directly from python arrays PTA( LMat/vec/…) to PTA(float/double), getting rid of the middle step? Plus if the user chages only some elements of the array, should we still make one call sending all the data. or should we make multiple call sendind only the data that is changed?

By the way this is my code in glShaderContext. I replaced SAT_float with SAT_vec/mat so it’s independent from the type (bool,half,int,double,float…), and the “shape” (vec1/2/3, mat1x1 etc…) of the parameter specified in the shader. I have already tested it and works fine for all the Cg numeric inputs ( also for a matrix bool1x1… )

  for (int i=0; i<(int)_shader->_ptr_spec.size(); i++) { 
    if (_shader->get_language() == Shader::SL_GLSL){
      GLCAT.error() << "ptr_anything parameter are supported only for Cg\n";
      release_resources(gsg);
      return;
    }

#ifdef HAVE_CG
    else if (_shader->get_language() == Shader::SL_Cg){
      const Shader::ShaderPtrSpec& _ptr = _shader->_ptr_spec[i];
      const Shader::ShaderPtrData* _ptr_data = gsg->fetch_ptr_parameter(_ptr);
      
      if (_ptr_data == NULL){ //should happen only if the input is not a PTA
        release_resources(gsg);
        return;
      }
      
      //check if the data must be shipped to the GPU
      if (!_ptr_data->_updated)
        continue;

      CGparameter p = _cg_parameter_map[_ptr._id._seqno];

      //Check if the size of the shader parameter and the input match
      int input_size = _ptr._dim[0] * _ptr._dim[1] * _ptr._dim[2];
      if (input_size != _ptr_data->_size){

        GLCAT.error() << "size of " <<  string(_ptr._id._name.c_str()) << " " << input_size << 
          " different from expected " <<  _ptr_data->_size;
        
        release_resources(gsg);
        return;
      }

      switch(_ptr_data->_type){
        case Shader::SPT_float:
          switch(_ptr._info._class){
            case Shader::SAC_scalar: cgSetParameter1fv(p,_ptr_data->_vectorf.p()); continue;
            
            case Shader::SAC_vector:
              switch(_ptr._info._type){
                case Shader::SAT_vec1: cgSetParameter1fv(p,_ptr_data->_vectorf.p()); continue;
                case Shader::SAT_vec2: cgSetParameter2fv(p,_ptr_data->_vectorf.p()); continue;
                case Shader::SAT_vec3: cgSetParameter3fv(p,_ptr_data->_vectorf.p()); continue;
                case Shader::SAT_vec4: cgSetParameter4fv(p,_ptr_data->_vectorf.p()); continue;
                default: LOG("Wrong type in SAC_vector") continue;
              }
            
            case Shader::SAC_matrix: cgGLSetMatrixParameterfr(p,_ptr_data->_vectorf.p()); continue;
            
            case Shader::SAC_array:{
              switch(_ptr._info._subclass){
                case Shader::SAC_scalar: cgGLSetParameterArray1f(p,0,_ptr._dim[0],
                                             _ptr_data->_vectorf.p()); continue;
                case Shader::SAC_vector:
                  switch(_ptr._dim[2]){
                    case 1: cgGLSetParameterArray1f(p,0,_ptr._dim[0],_ptr_data->_vectorf.p()); continue;
                    case 2: cgGLSetParameterArray2f(p,0,_ptr._dim[0],_ptr_data->_vectorf.p()); continue;

... 
          }
case Shader::SPT_double: 

... same as above ...

I’ll get rid of LOG(…) when it will be finished.

memecss · February 26, 2010, 7:21am

I am still thinking how to have generic code and performances , and at the moment I can’t find what’s the better solution. My ideas are:

  struct ShaderPtrData{
  public:
    ShaderPtrType _type;
    int           _size;
    bool          _updated;
    float*        _ptrf;
    double*       _ptrd;

  private:
    PTA(float)    _pta_float;
    PTA(double)   _pta_double;
  
  //constructors
  public:
    ...

    ShaderPtrData(const PTA(float)& ptr):
      _pta_double(PTA(double)::empty_array(0)),
      _pta_float(ptr),
      _ptrf((float*)ptr.p()),
      _ptrd(NULL),
      _type(SPT_float),
      _size(ptr.size()),
      _updated(true){};
      
      ...

So as you can see this is setting a float*/double* equals PTA(…).p(). I know PTAs handles memory them-self but it should not be a problem since they pointer and PTA have the same scope. I should set them to be const but I want to allow to change only some elements of the arrays. This is good because I am not allocating memory, but it gets ugly if we add lots of types (ie PTA(LVecBase/LMat…).

The other idea would be simply having only

  public:
    ShaderPtrType _type;
    int           _size;
    bool          _updated;
    float*        _ptrf;
    double*       _ptrd;

and memcpy the data in it, getting rid of PTAs.
This has the opposite problem. Clean code but of waste memory. That should not be a big deal for simple matrices or vector but skinning on the GPU (that is one of the reasons I am adding this inputs) could become quite expensive.

drwr · February 26, 2010, 4:45pm

Hmm, yes, I see the problem. My inclination would be to go with the former solution in this case, even though it’s uglier, because at this level the performance makes a difference.

David