time for some more terrifying questions.
i tried to write a small artoolkit sample to be included in panda which is not going very smooth at all.
first i found that WebcamVideo is totaly empty under linux. it entirely relies on DirectShow under windows and doesnt even contain a single line of code for linux.
so i had to use openCV. which was broken under linux,too. so i had to grep the 1.1.0-pre version of the opencv source , compile it, and compile panda against it,too.
so i got the pictures of my webcam showing up on textures.
then i tried to use ARToolKit, which was broken,too under linux (actually all non-windows systems since w32 is the only one using BGRA by default), wrong color-space was compiled into the .a files. so i had to change it (thx to pro) and compile panda against the new artoolkit and new opencv.
now opencv delivers video, and artoolkit works on textures i load from images.
but when i try to feed an opencv texture to artoolkit it crashes with an assertion error.
i would provide a full snipsset but it requres additional files, and fixed up panda compiles so i guess few people are interested in it.
....
.....
tex = OpenCVTexture()
tex.fromCamera(0)
toolkit = ARToolKit.make(base.cam,Filename("./camera_para.dat"),1)
toolkit.attachPattern(Filename("./patt.hiro"), box)
#now thats the line where it crashes
toolkit.analyze(tex)
output:
the analyze works fine with “standart” textures
Assertion failed: tex->has_ram_image() at line 260 of panda/src/grutil/arToolKit.cxx
Traceback (most recent call last):
File "sample2.py", line 32, in <module>
toolkit.analyze(tex)
AssertionError: tex->has_ram_image() at line 260 of panda/src/grutil/arToolKit.cxx
help is very welcome since it looks like the last stone missing in the artoolkit puzzle.
i traced the error through panda’s source code. i ended up in the opencvtexture.cxx’s update_frame function. from what i understood its the only function that actually fills the _ram_images with data. since there was none. i spawned cerr statements at several points in the file.
it looks like update_frame never ever gets executed.
this would also explain why my texture is not updated automatically. i can capture a single frame with fromCamera(0) but it wont change unless i call fromCamera again.
so i came to the conclusion there has to be something wrong with videoTexture.I and its consider_update() .
anyone of the gods-of-panda-source can confirm this? ffmpeg texture relies on the same frame_update() so if that’s indeed the problem that might be affected aswell.
sorry i can’t help too much, but i’m very fond of both artoolkit and opencv, so I hope you get this working…
when you say broken you just mean panda’s wrapper, not artoolkit and opencv themselves, right?
i’m curious about the way this works in panda - the artoolkit object doesn’t do the capture itself, you have to feed it the frames from elsewhere? or is that because of the colorspace issue you mentioned?
artoolkit and opencv itself work ok. its just the passing around of imagedata in panda which seems broken for now.
edit- to be more precise. the updating of the data seems to be never called. so there never is any data to work with anyway.
Well, ARToolKit checks has_ram_image on the OpenCVTexture (which inherits from VideoTexture). VideoTexture::do_has_ram_image looks sortof like this:
Check the last frame that the ram image got updated.
If it's not the current frame, return false.
So it might return false when there’s actually a ram image.
Now, what makes this ram image?
There’s this function update_frame, right? Well, it only gets called once, on the first frame. (IMHO, it should also get called on a from_camera call.)
So, since update_frame is not called every frame, has_ram_image returns false on all other frames.
Now, ARToolKit calls this has_ram_image and asserts if it is not true. We should uncomment that assertion.
But there’s still the problem of update_frame not getting called every frame – can you comment on how that’s supposed to happen, David?
It sounds like bit-rot to me. The internal API to rendering Textures has changed a bit over the past year. Since I didn’t have the OpenCVTexture or ARToolkit code in my build, I wasn’t able to preserve their functionality. I’m not entirely sure how they were ever supposed to work, but surely it shouldn’t be difficult to make them work again.
well we are not sure how it worked at all
with the assertion commented out we can get a one-time ram_image.
its not updated by frame_update.
when trying to pass the image to artoolkit it doesnt work cause artoolkit require a RGBA or BGRA (in any case 4 color components) image.
changing the texture format either causes another error or it resets it back to 3-component (depends on whether you do setFormat before or after fromCamera() )
working around this by getting a pnm-snapshot of the current frame or saving it out to disk doesnt work either since those functions are broken for opencv textures,too.
any chance to see this fixed? i mean. artoolkit is really nice but it looks like the only platform it even “might” work is windows.
opencv is as-good-as-broken.
webcamVideo is an empty function for all non-windows os-es.
I’m looking into fixing OpenCV right now. The main problem is that update_frame gets called only once, and not every frame.
drwr, could you give me some info on how this update_frame is actually supposed to be called? I believe it’s called in consider_update, but how is that supposed to be called? I’ve been checking the logs of ffmpegTexture but don’t see changes there that aren’t changed in openCVTexture as well.
I honestly don’t know. The update_frame() method doesn’t sound familiar to me. But OpenCVTexture is a very old implementation of a video texture, and it might not have worked for years. Does ffpmegTexture work? What does it do differently that doesn’t require a call to update_frame()?
FfmpegTexture also implements update_frame.
I have a theory: maybe OpenCV has only been used in the past for videos, and not for webcam input, and update_frame only gets called when you actually play a video? I’ll try to trace back the call paths.
PS. Just committed several fixes to ARToolKit and OpenCV that make it possible to use them together, and yay, it actually works!
