Here’s a proposal. There are some details I haven’t worked out yet, but it’s about 90% complete, and I think comments would be useful at this point.
Filters inherit from a Filter class, which defines the API that CommonFilters talks to. Each subclass describes a particular type of filter (CartoonInkFilter, BlurSharpenFilter, …). These can be implemented in their own Python modules. Very short and simple filters, for which a separate module would be overkill, can be collected into one MiscFilters module.
The new modules will be placed in a subdirectory to avoid confusing the user, as CommonFilters and FilterManager will remain the only classes meant for public use. For example, the filter modules could reside in direct.filter.internal, direct.filter.impl or some such.
A pipeline will be added, in order to correctly apply filters that need the postprocessed (or post-postprocessed etc.) scene as input*. Currently BlurSharpen is the only one that needs this, but that will likely change in the future.
(* Strictly speaking, the critical property is whether the filter needs lookups in its input colour texture at locations other than the pixel being processed.)
To implement the pipeline, the control logic of CommonFilters itself will be split into two modules. The first is CommonFiltersCore, which performs the task of the low-level logic in the current CommonFilters, providing shader synthetization for a single stage of the pipeline. The synthesis will be implemented in a modular manner, querying the configured Filters. (Details further below.)
The second module is a backward-compatible CommonFilters, which provides the user API (the high-level part of current CommonFilters), and takes care of creating the necessary stages and assigning the configured filters to them. That is, the user configures all filters in a monolithic manner (in the same way as in the current version), and then CommonFilters creates the necessary CommonFiltersCore instances and splits parts of the configuration to each of them in the appropriate manner. This allows keeping the core logic simple, as it does not need to know about the pipeline.
To support multiple stages, CommonFilters and FilterManager will be extended to capture buffers, in addition to the current mode of operation, where they capture a window with a camera.
Adding the buffer capture feature has a desirable side effect: the user will be able to pipe together CommonFilters (the high-level object) instances with custom FilterManager filters. For example, the scene may first be processed by some CommonFilters, then by some custom filters, and finally more CommonFilters. This gives an extremely flexible modular design also from the user’s perspective, making CommonFilters and the custom filter mechanism complement each other (instead of being alternatives, as in the current version).
The pipeline consists of stages. Roughly speaking, a stage is an ordered collection of filters that can be applied in one pass.
Stages are represented in the high-level CommonFilters class by CommonFiltersCore objects kept in an ordered list.
Each stage has an input color texture. Depth and aux textures are always taken from their initial source. (It would be possible to support processing these, too, by allowing the fshader to output multiple textures. Currently it’s not needed.)
Each stage has an output color texture.
The input to the first stage is the input scene or texture that was provided by the user to CommonFilters.
For subsequent stages, the pipeline connects the output of stage k to the input of stage k+1.
The output from CommonFilters is the output of the last stage.
Each filter derived from the Filter class:
must provide a list of names for internal textures, if it needs any. CommonFiltersCore will use this to manage the actual texture objects.
must define which textures (color, depth, aux, any internals) it needs as input in the compositing fshader, and for which of those it needs texpix.
must declare any custom parameters and their types (the k_variables that can be configured at runtime using setShaderInput()). These are appended by CommonFiltersCore to the parameter list of the compositing fshader.
must declare a sort value, which determines the filter’s placement within the pipeline stage it is placed in. This determines the placement of the fshader code snippet within the compositing fshader. (Note that filters that do not require internal intermediate stages, or texture lookups other than the pixel being processed, can be implemented using only an fshader code snippet.)
must provide a function that, given a FilterConfig for this particular type of filter, synthesizes its code snippet for the compositing fshader. This function compiles in the given values of compile-time parameters. The fragment shader code must respect any previous modifications to o_color to allow the filter to work together with others in the same compositing fshader.
must provide a function that compares oldconfig and newconfig, and returns whether a shader recompile is needed. (Only each filter type itself knows which of the parameters are runtime and which are compile-time.)
must provide a function to apply values from newconfig to runtime parameters. This is called at the end of reconfigure() in CommonFiltersCore.
may optionally implement a function to set up or reconfigure any internal processing stages. This includes synthesizing shaders for the internal intermediate textures. (The default implementation is blank.)
One detail I haven’t decided on is the initial value of o_color before the first filter in a stage is applied. There is a design choice here: either always make the compositing fshader to initialize o_color = tex2D( k_txcolor, l_texcoord.xy ), or alternatively, require exactly one of the filters registered to a stage (in high-level CommonFilters) to declare itself to be “first”, in which case that filter must in its fshader snippet write the initial value to o_color. The latter approach is more general (allowing for optimality when the default is wrong), but the first one is simpler, and often sufficient.
Another open question is where to declare which filter belongs to which stage in the high-level logic. The simplest possibility is to identify stages by names, a valid list of which would be provided in the high-level CommonFilters. This would allow the information to reside in the Filter subclasses themselves. CommonFilters itself would only need to be updated if a new stage is required. The stage information would be spread out across the individual Filters, which can be considered as both an advantage (everything related to a particular type of filter in the same place) and as a drawback (hard to get the big picture about filter ordering, because that requires checking each Filter subclass module and keeping notes).
The other possibility I have thought of so far is to hard-code the stage for each known subclass of Filter in the high-level CommonFilters. This would keep that information in one place (making it easy to understand the overall ordering), but this solution requires updating CommonFilters whenever a new type of filter is added. Also arguably, the stage information is something that logically belongs inside each type of filter.
HalfPixelShift is a special case, which does not conform to this filter model. It could be implemented as a half-pixel shift option to CommonFiltersCore. Enabling this would cause CommonFiltersCore to emit the code for HalfPixelShift in the compositing vshader. It would be enabled for the first stage only (in the high-level CommonFilters).
So, that’s the current plan. Comments welcome.
Regarding the buffer capture, so far I’ve gotten CommonFilters to initialize from a buffer by manually creating a buffer and setting up a quad and a camera for it, but I’m not sure whether this is the right way to do it. Probably not - it seems FilterManager already internally creates a quad and a camera. To prevent duplication, it needs to be able to read a render-into-texture input buffer (with color, aux, depth textures) directly.
I’d appreciate any pointers as to the correct approach to do this
I don’t see why. That seems unnecessarily restrictive. People should be able to create their own instances of the individual Filter classes, inherit from them, etc. Part of the flexibility that this overhaul would offer is that it would allow people to customise CommonFilters with their own filters by creating their own Filter class and adding it.
I imagine that CommonFilters can store a list of filters with a sort value each that would determine in which order they get applied in the final compositing stage…
I imagine methods like setVolumetricLighting() to become simple stubs that call something like addFilter(VolumetricLightingFilter(*args, **kwargs)).
Makes sense. By a “single stage”, you mean a part of the final compositing filter, or an individual render pass (like the blur-x and blur-y passes of the blur filter)? These are two separate concepts, though I suppose they don’t strictly need to be.
Ah, so CommonFiltersCore represents a single filter pass? (Can we call it something like FilterPass or FilterStage then?) Can we let the Filter class set up and manage its stages rather than the CommonFilters? It does sound like something that would be managed by each filter, although I suppose some filters might share a filter pass (ie. blur and bloom may share an initial blur pass). Hmm.
I don’t understand what you mean by “capturing a buffer”, could you please explain that? You can already use FilterManager with a buffer, if that’s what you meant, but I don’t quite understand the necessity of that.
Could the user achieve the same thing by subclassing Filter and adding this Filter to the same CommonFilters object?
Then I think that FilterStage would be a far more representative term, don’t you think?
One thing I don’t quite understand - is a stage a render pass by itself, or a stage in the final compositing shader?
Not all stages need an input color texture. SSAO, for instance, does not.
I think FilterConfig is obsoleted by the new Filter design, since each Filter can just take all of the properties in the constructor via keyword arguments, and have properties or setters that invalidate the shader when they are modified depending on the property. Each setter of a particular Filter could either update a shader input or mark the shader as needing to be regenerated.
I think that each filter could possibly be a Cg function with the arguments it needs passed to it for better organisation.
You could have a filter stage that’s added by default with a negative sort value with its only purpose being to set o_color, which is always applied first.
I agree that this probably belongs in the the individual Filter classes.
I think HalfPixelShift should be a global setting in CommonFilters and not a filter at all.
I think at this point it would help to hack up some pseudo-code that kind of shows how the systems work together and perhaps showing a example filter while skipping over the details. It would help to get a good overview and help me to understand your design better.
Maybe I should explain what I was trying to achieve.
The idea was that it should be easy to learn to use the CommonFilters system by reading the API documentation. At least I have learned a lot about Panda by searching the API docs.
If all the modules are placed in the same directory as CommonFilters itself, there will be lots of modules in the same place, and finding the interesting one becomes difficult.
I agree that flexibility is desirable.
Adding it where? In their local copy of the Panda source tree?
Hmm, this would make it easier to contribute new filters to Panda, which is nice.
Yes, that is part of the solution. But there are two separate issues here:
First is where to store the sort values. If I understood correctly, we seem to agree that this information belongs in the Filter subclasses.
Secondly, there are some filter combinations that cannot be applied in a single pass. BlurSharpen and anything else is one such combination - the blur will not see the processing from the other filters applied during the same pass.
Yes, something like that.
Thanks for asking (I’m sometimes very informal about terminology). By “stage of pipeline”, I meant a render pass.
But that doesn’t capture the idea strictly, either. From the viewpoint of the pipeline, the important thing to look at are the input textures needed by each filter.
Filters that share the same input textures (down to what should be in the pixels), and respect previous modifications to o_color in their fshader code, can work in the same pass. I think it’s a potentially important performance optimization to let them do so, so that enabling lots of filters does not necessarily imply lots of render passes.
Some filters may have internal render passes (such as blur), but to the pipeline this is irrelevant. Blur works, in a sense, as a single unit that takes in a colour texture, and outputs a blurred version. The input colour texture is the input to that pass in the pipeline where the blur filter has been set.
If the aim is to blur everything that is on the screen, the blur filter must come at a later render pass in the pipeline, so that it can use the postprocessed image as its input.
My proposal was that the core synthesizes code for a single “pipeline render pass”, so that the pipeline setup can occur in a higher layer (creating several, differently configured instances of the core).
Yes, we can change the name to something sensible
Any internal stages (passes) (e.g. blur-x and blur-y) are indeed meant to be handled by each subclass of Filter.
About sharing passes in general, I agree. That is the reason to have a code generator that combines applicable filters to a single pass in the pipeline.
About blur and bloom specifically, I think they belong to different passes, because the effects they reproduce happen at different stages in the image-forming process.
I would like to set up the ordering of the filters as follows:
full-scene antialiasing (if added later)
CartoonInk, to simulate a completely drawn cel
optical effects in the scene itself (local reflection (if added later), ambient occlusion, volumetric lighting in that order)
optical effects in the lens system (bloom, lens flare)
film or detector effects (tinting, desaturation, colour inversion)
computer-based postprocessing (blur)
display device (scanlines)
debug helpers (ViewGlow)
Keep in mind that e.g. chromatic aberration in the lens should occur regardless of whether the result is recorded on colour or monochrome film.
Also note that these categories might not be exhaustive, might not correspond directly to render passes, and in some cases it can be unclear which category a given filter belongs to. For example, I tend to think of blur as a computer-generated postprocessing effect (requiring a complete “photograph” as input), but it could also represent the camera being out of focus, in which case it would come earlier in the pipeline (but definitely after CartoonInk and scene optical effects). I’m not sure what to do about such cases.
(Bloom, likewise, may be considered as a lens effect (the isotropic component of glare), or as a detector effect (CCD saturation). Maybe it is more appropriate to think of it as a lens effect.)
Finally, note that currently, only lens flare supports chromatic aberration. I think I’ll add full-screen chromatic aberration and vignetting to my to-do list, to approach a system that can simulate lens imperfections.
There are two use cases I’m thinking of.
First is daisy-chaining custom filters with CommonFilters. People sometimes use FilterManager to set up custom shaders, but the problem is that if you do that, it is not easy to apply CommonFilters on top of the result (or conversely, to apply your own shaders on top of what is produced by CommonFilters). When you apply either of these, you lose the camera, and can no longer easily set up the other one to continue where the other left off.
For a thought experiment, consider the original lens flare code by ninth (attached in Lens flare postprocess filter), and how you would go about applying CommonFilters to the same scene either before or after the lens flare. If I haven’t missed anything, currently it is not trivial to do this.
The second case is a scene with two render buffers doing different things, which are both postprocessed using CommonFilters, then rendered onto a quad (using a custom shader to combine them), and then the final quad is postprocessed using CommonFilters. There is a code example in my experiment on VolumetricLighting with cartoon-shaded objects: [Sample program] God rays with cartoon-shaded objects which probably explains better what I mean.
The thing is that at least in 1.8.1, setting up the combine step is overly complicated:
If you have a camera, it is just one line to call FilterManager to set up the render-into-quad, but if you don’t (because CommonFilters took it), you need to do more API acrobatics to create one and set up the render-into-quad manually.
EDIT: Also, then FilterManager (or CommonFilters when it calls FilterManager internally) goes on to obsolete the manually created quad and camera, creating another quad and another camera. It would be nice to avoid the unnecessary duplication. I don’t know if it affects performance, but at least it would make for a cleaner design.
Then, in both cases, we set up the combining shader
self.finalfilters = CommonFilters(base.win, quadcamera)
self.finalfilters.setBlurSharpen() # or whatever
though here, now that I think of it, I’m not sure how to get the quad camera in the case where FilterManager internally creates it.
In summary, what I’m trying to say is that I think these kinds of use cases need to be more convenient to set up
The difficulty in that approach is that the user needs to understand the internals of CommonFilters, in order to be able to set up the pipeline pass number and sort-within-pipeline-pass priority correctly, in order to make CommonFilters insert the shader at the desired step in the process. Especially, the user must know which pipeline pass the shader can be inserted into (so that it won’t erase postprocessing by other filters; consider the blur case).
In addition, the user-defined shader must then respect the limitation that within the same pipeline pass, each fshader snippet must respect any previous changes to o_color. I think it is error-prone to require that of arbitrary user code, and especially, this makes it harder just to experiment with shaders copied from the internet.
Also, the user then needs to conform to the Filter API. If the user wants to contribute to CommonFilters, that is the way to go. But for quick experiments and custom in-house shaders, I think FilterManager and daisy-chaining would be much easier to use, as then any valid shader can be used and there are no special conventions or APIs to follow.
As mentioned above, I was speaking of a render pass (but with the caveats mentioned).
Of the code for different filters in the compositing shader, I used the term “snippet” as I didn’t have anything better in my mind
That is another way to do it. May be cleaner.
Does this bring overhead? Or does the compiler inline them?
Also - while I’m not planning to go that route now - Cg is no longer being maintained, so is it ok to continue using it, or should we switch completely to GLSL at some point?
That’s one way of applying the default.
But how likely is the default to be wrong, i.e. do we need to take this case into account?
EDIT: Aaaa! Now I think I understand. If the default is wrong, then override this default filter stage somehow? E.g. sort=-1 means the output colour initialization stage, and if a stage with that sort value is provided by the user, that one is used, but if not, then the default one is used.
Wow, that’s quite a bit more than some simple pseudo-code. Thanks.
It looks great to me! A few minor comments.
Instead of getNeededTextures, I would suggest that there is instead a setup() method in which the Filter classes can call registerInputTexture() or something of the sort. The advantage of this is that we can later extend which things are stored about a texture input by adding keyword arguments to that method, without having to change the behaviour in all existing Filter implementations. It seems a bit cleaner as well. The same goes for getCustomParameters.
getNeedGlow seems a bit specific. Can we instead store a bitmask of AuxBitplaneAttrib flags?
I’m not quite sure I understand this stage idea. Is the “stage” string one of a number of fixed built-in stages? Are the different stages hard-coded? Can you explain to me in simple terms what exact purpose the stage concept serves?
I’m not sure if all of those methods need getters - it seems that some of them can simply remain a public member, like sort and needs_compile. I think sort can be a member with a filter-specific default value, but that can be changed by the user.
I think the strange inspection logic in setFilter has to go. We should keep it simply by either allowing someone to add a filter of a certain type more than once (even if that doesn’t make sense), or raising an error, or removing the old one entirely.
Just FYI, cmp= in sort() is deprecated and no longer supported in Python 3. Instead, you should do this:
self.filters.sort(key=lambda f: f.sort)
where Filter stores a self.sort value.
I think there is no reason to keep CommonFilters an old-style class. Perhaps CommonFilters should inherit from FilterPipeline?
Ah, this indeed sounds more extensible. Let’s do that.
Yes, why not.
The other day, I was actually thinking that SSLR will need gloss map support from the main render stage, and this information needs to be somehow rendered from the material properties into a fullscreen texture… so, a general mechanism sounds good
In this initial design, yes and yes, but the idea is that it is easy to add more (when coding new filters) if needed.
I’m not completely satisfied by this solution, but I haven’t yet figured out a better alternative which does not involve unnecessary bureaucracy at call time.
In short, the stage concept is a general solution to the problem of blur erasing the output of other postprocessing filters that are applied before it.
Observe that the simplest solution of applying blur first does not do what is desired, because then the scene itself will be blurred, but all postprocessing (e.g. cartoon ink) will remain sharp.
The expected result is that blur should apply to pretty much everything rendered before lens imperfections (or alternatively, to pretty much everything except scanlines, if blur is interpreted as a computer-based postprocess).
As for the why and how:
As you know, a fragment shader is basically an embarrassingly parallel computation kernel, i.e. it must run independently for each pixel (technically, fragment). All the threads get the same input texture, and they cannot communicate with each other while computing. The only way to pass information between pixels is to split the computation into several render passes, with each pass rendering the information to be communicated into an intermediate texture, which is then used as input in the next pass.
The problem is that with such a strictly local approach, some algorithms are inherently unable to play along with others - they absolutely require up-to-date information also from the neighbouring pixels.
Blur is a prime example of this. Blurring requires access to the colour of the neighbouring pixels as well as the pixel being processed, and this colour information must be fully up to date, to avoid erasing the output of other postprocessing algorithms that are being applied.
I’m not mathematically sure that blur is the only one that needs this, and also, several postprocessing algorithms (for example, the approximate depth-of-field postprocess described in http.developer.nvidia.com/GPUGem … _ch28.html) require blurring as a component anyway. Thus, a general solution seems appropriate.
The property, which determines whether another stage is needed, is the following: if a filter needs to access its input texture at locations other than the pixel being rendered, and it must preserve the output of previous postprocessing operations also at those locations, then it needs a new stage. This sounds a lot like blur, but dealing with mathematics has taught me to remain cautious about making such statements
(For example, it could be that some algorithm needs to read the colour texture at the neighbouring pixels just to make decisions, instead of blurring that colour information into the current pixel.)
One more note about stages - I’m thinking of adding automatic stage consolidation, i.e. the pipeline would only create as many stages as are absolutely needed. For example, if blur is not enabled, there is usually no reason for the post-blur filters to have their own stage.
More about this later.
Ok. May be cleaner.
On this note, I’ve played around with the idea of making the filter parameters into Python properties. This would have a couple of advantages.
First, we can get rid of boilerplate argument-reading code in the derived classes. The Filter base class constructor can automatically populate any properties (that are defined in the derived class) from kwargs, and raise an exception if the user is trying to set a parameter that does not exist for that filter (preventing typos). This requires only the standard Python convention that the derived class calls super(self.class, self).init(**kwargs) in its init.
Secondly, as a bonus, this allows for automatically extracting parameter names - by simply runtime-inspecting the available properties - and human-readable descriptions (from the property getter docstrings).
That sounds good. Let’s do that.
Maybe stage should be user-changeable, too. (Referring here to the fact that for some filters (e.g. blur), the interpretation of what the filter is trying to simulate, affects which stage it should go into.)
The only purpose here was to support the old API, which has monolithic setThisAndThatFilter() methods that are supposed to update the current configuration.
If this can be done is some smarter way, then I’m all for eliminating the strange inspection logic
Ok. Personally I’m pretty particular about Python 2.x (because of line_profiler, which is essential for optimizing scientific computing code), but I agree that Panda shouldn’t be.
I’ll change this to use the forward-compatible approach.
Maybe. This way, it could simply add a backward-compatible API on top of FilterPipeline, while all of the functionality of the new FilterPipeline API would remain directly accessible. That sounds nice.
I’ll have to think about this part in some more detail.
EDIT: updated the attached code. Code generation and HalfPixelShift are now done. EDIT2: fixed bug in code enabling HalfPixelShift and some erroneous comments. Attachment updated. EDIT3: fixed some comments and asserts. EDIT4: update task mechanism added; it is now a registrable for each individual Filter. ScanlinesFilter provides an example. EDIT5: the attachment in this post is the last version before the module split; it is now obsolete. See the later post, including code that has been split into modules.
A first version of the CommonFilters re-architecture is almost complete.
I still need to split the code into modules and add imports, and port most of the existing filters (including my new inker) over to the new architecture, but the infrastructure should now be in place.
I expect to get to the testing phase in a day or two.
Multi-passing with automatic render pass generation based on filter properties. Filters are assigned to logical stages (corresponding to steps in the simulated image-forming process), and the pipeline figures out dynamically how many render passes to create and which stages to assign to each. This allows e.g. blur to see cartoon outlines, opening up new possibilities.
Allows mixing filters provided with Panda and custom filters in the same pipeline, as long as the custom filters are coded to the new Filter API (which is the same API the internal filters use). The API aims to be as simple as possible. This also makes it easier to contribute new filters to Panda.
Filters may define internal render passes, allowing filters with internal multi-pass processing. (This is just to say that the new architecture keeps this feature!)
Highly automated. Create run-time and compile-time filter properties with one-liners (or nearly; most of the length comes from the docstring). Assign a value to a filter property at run-time, and the necessary magic happens for the new value to take effect, whether the property represents a shader input or something affecting code generation.
Runtime-inspectable; filters have methods to extract parameter names, or parameter names and their docstrings. You can also get the current generated shader source code from each FilterStage by just reading a property.
Object-oriented architecture using new-style Python objects. Using inheritance it is possible to create specialized versions of filters (to some degree).
Exception-based error handling with descriptive error messages to ease debugging.
The texcoord handler is based on the latest version in CVS, but it now handles texpad and texpix separately (to cover the case where HalfPixelShift is enabled for non-padded textures; in this case the vshader needs texpix but no texpad).
This source code was retrieved from the framework by:
for stage in mypipeline.stages:
In the Panda spirit, you can ls() the FilterPipeline to print a description:
FilterPipeline instance at 0x7f9c4a655f50: <active>, 1 render pass, 1 filter total
Scene textures: ['color']
Render pass 1/1:
FilterStage instance '[LensFocus]' at 0x7f9c3a3cac50: <2 filters>
Textures registered to compositing shader: ["blur1 (reg. by ['BlurSharpenFilter'])", "color (reg. by ['StageInitializationFilter'])"]
Custom inputs registered to compositing shader: ["float k_blur_amount (reg. by ['BlurSharpenFilter'])"]
StageInitializationFilter instance at 0x7f9c3a3cac90
BlurSharpenFilter instance at 0x7f9c3a3caad0; 2 internal render passes
Internal textures: ['blur0', 'blur1']
(If it looks like the framework can’t count, rest assured it can - the discrepancy in the filter count is because StageInitializationFilter is not a proper filter in the pipeline, but something that is inserted internally at the beginning of each stage. Hence the pipeline sees only one filter, while the stage sees two.)
The legacy API is a drop-in replacement for CommonFilters - the calling code for this test was:
(The nonstandard path for the import is because these are experimental files that are not yet in the Panda tree. It will change to the usual “from direct.filter.CommonFilters import CommonFilters” once everything is done - so existing scripts shouldn’t even notice that anything has changed.)
Now, I only need to port all the existing filters to this framework, and then I can send it in for review
Latest sources attached. There shouldn’t be any more upcoming major changes to the framework itself. What will change is that I’ll add more Filter modules and update the legacy API (CommonFilters) to support them. CommonFilters190_initial_working_version.zip (112 KB)
I suppose that’s on par for the course when a large new release is coming up
In the meantime, I can proceed with porting the filters (both the old and the new ones). I’ll post a new version on the weekend.
There are a couple of things for which specifically I’d like comments - since this subsystem is pretty big, it might be easier to spell them out here:
Legacy API and new filters, and new options for old filters (CartoonInk)? Support them there (so that legacy scripts require only minimal changes to add support for new filters), or leave them as exclusive to the new API (from the user’s perspective, the new API allows even simplifying the calling code, since it is now possible to change parameter values selectively instead of always sending a christmas tree; but this is more work for the user)?
Should FilterManager be modified to support partial cleanups, or is the current solution (that almost always rebuilds the whole pipeline) actually simpler? I’d like to have a system that rebuilds only the changed parts - hierarchical change tracking is already there, so for most cases this would be simple if only FilterManager allowed it. (But there are exotic cases, e.g. changing VolumetricLighting’s “source” parameter, or toggling the depth-enabled flag of the new CartoonInk. These affect which textures are required - and if the scene texture or aux bits requirements change, then it’s off to a pipeline rebuild anyway, because FilterManager.renderSceneInto() must be re-applied with the changed options.)
I’m trying to keep this as hack-free as possible, but there are some borderline cases. I’d like to prioritize power and ease of use - but if you see something that looks like a hack and have an idea how to achieve the same thing cleanly, I’d like to know
By the way, I got rid of the inspection logic in setFilter() - now the logic to apply recompiles only when necessary resides in the property setters, where I think it belongs. FilterPipeline still has a setFilter() that either adds or reconfigures a filter (depending on whether it already exists in the pipeline), but now its implementation is much simpler.
Ah, and for now, only one instance per filter type is supported in any given pipeline instance - supporting multiple instances of the same filter in the same pipeline is a bit tricky (details in the code comments and README; in short, this requires some kind of name mangling).
I think the idea of the CommonFilters system was very good - having certain postprocessing operations available that can be simply switched on and configured, in any combination. I find it similar in spirit to the main shader generator: 99% of the time, there is no need to write custom shaders.
The aim of the new FilterPipeline framework is something similar for postprocessing filters. What it adds to the postprocessing system is maintainability (keeping code complexity in check as more filters are added) and extensibility (so that people in the community can write their own filters that plug in to the pipeline). Also, the automatic render pass generator significantly extends the degree how well the different filters play together.
From a user perspective, the interesting part will be new filters. As the first step, I’ll be adding the ones I’ve already coded, i.e. desaturation, scanlines, and lens distortion.
I’ve also been eyeing ninth’s SSLR (local reflection) implementation, which is very cool (and he’s ok with including it to Panda). The latter may require changes to other parts of Panda to supply a fullscreen gloss map texture to the postprocessing subsystem, but it should be possible to do. We already have SSAO, so SSLR would be a nice addition to support more high-end visual effects out of the box
I also bumped into an independent implementation of an early FXAA (Fast approximate antialiasing) version that was “feel free to use in your own projects” ( horde3d.org/wiki/index.php5?titl … que_-_FXAA ), so I think I’ll be adding that, too. The fshader is just a screenful of code, so it’s very simple. It would be a nice alternative for smoothing both light/dark transitions and object edges in cartoon-shaded scenes, as FXAA is basically an intelligent anisotropic blur filter. It may also be useful as a very cheap filter for general fullscreen antialiasing on low-end hardware.
There’s also SMAA (Enhanced subpixel morphological antialiasing), which is available under a free license, but its implementation is much more complex, and I haven’t yet investigated whether it’s possible to integrate into the new pipeline. See github.com/iryoku/smaa
I’d also very much like to add a depth-of-field (DoF) filter. While no perfect solution is possible using current hardware, the algorithm explained in GPU Gems 3 is pretty good for a relatively cheap realtime filter. See the article, which also contains a nice overview of possible techniques and references to papers discussing them: http.developer.nvidia.com/GPUGem … _ch28.html
Then, there is something that could be improved in the existing filters - for example, I think blur would look more natural using a gaussian kernel. Also, it doesn’t yet use the hardware linear interpolation to save GPU cycles, so the current implementation is not optimally efficient. This was covered in a link posted earlier, rastergrid.com/blog/2010/09/effi … -sampling/
And further, the blur kernel size could be made configurable, to adjust the radius of the effect. For a small blur, it is possible to sample 17 pixels using just five texture lookups in a single pass - and that’s including the center pixel. A diagram of the stencil can be found in the DoF article linked above, in subsection 28.5.2 - it’s pretty obvious after you’ve seen it once.
Added most of the existing filters and fixed some bugs (notably, buffer creation order, and a corner case in the logical stage merging logic (old version failed if all stages were non-mergeable)). Bloom and AmbientOcclusion turned out to need some new features, too (the “paramproc” argument in Filter.makeRTParam(), and “internalOnly” scene textures, respectively).
New version attached.
Hopefully, I’ll get CartoonInk (the old one) and VolumetricLighting done tomorrow; then this is ready for adding in the new filters. (Though I do need to comb through all the comments and docstrings to make sure everything is up to date.) CommonFilters190_more_filters_and_bugfixes.zip (136 KB)
Well, those filters are not in yet, but I did make the code generator a bit cleaner, eliminating some corner cases where unneeded variables were passed to the filter functions. (E.g. ScanlinesFilter needs only texpix for the colour texture, not the actual colour texture (except for the current pixel in the “pixcolor” variable); and StageInitializationFIlter does not need the “pixcolor” argument, because it initializes pixcolor.)
Also, now the code handling the registerInputTexture() metadata should be easier to read. Updated version attached.
Finally, one question, mainly to rdb - how is the “source” parameter in VolumetricLighting (in CVS) supposed to work? I’m tempted to leave that parameter out, since if I understand the code in CVS correctly, its current implementation will only work if the referred texture happens to be one of those already passed to the compositing fshader for other reasons.
Also, related to this, look at line 170 of CommonFilters.py in CVS - I think it should be needtex.add(…), like the others, instead of needtex[…] = True?
Providing a general mechanism for supplying any user-given texture to the compositing fshader is something I haven’t even considered yet, since none of the other filters have happened to need that. Might be possible to do by adding more options to registerInputTexture(), if needed.
But I’m not sure if doing that would solve volumetric lighting. If I’ve understood correctly based on my reading of GPU Gems 3, there are only two ways to produce correct results from this algorithm - a) use a stencil buffer; or b) render a black copy of the scene (with only the light source stand-in sprites rendered bright), invoke VolumetricLighting on that, render another copy of the scene normally, and finally blend those additively to get the final result.