tamapolis

First a little comment about RenderToTexture. My RTs are based in FBOs (Frame Buffer Objects), thats a feature relatively new on OpenGL and not well supported by all the graphics card.

Yesterday I was testing my app under OSX where I have everything installed (I used Macport to install python and all the libraries) and it crashed because the FBOs were not supported. Thats impossible because it is a MacBook and I know the card supports it, but for some reason the OpenGL functions relative to FBOs are not pointing to the right place on the driver, I don’t know what to do to solve it.

I also tested it in my other computer which runs Windows 7, and pygame crashes due to a problem with MSVCR71.dll, I guess is a SDL problem and Visual Studio, the funny part is that it works when I run the script from python IDLE. Weird. So at the end using python and opengl is not as cross platform as I thought.

My friend Miguel Angel pointed out also something curious, he had the same problem as Bastiaan when calling one of the OpenGL function, but that problem didn’t occour to me, it has to be something with the versions we are using, so I guess the Pyopengl API is not so solid and it is changing between versions more than what I would like, that could be a problem in the future, we will see…

PostFX

Today I’m gonna try to code a simple PostFX shader to see if I’ve got the basics.

The wrapper I did for the shaders is too simple right now, it doesnt support to upload textures to the shader, and that is mandatory, so thats my first enhacement. Also I haven’t test loading and showing a texture, indeed I coded the RT class without having a Texture class first (usually RT inherits from Texture).

I was expecting to find a class that encapsulates all the boring part of a texture, like texture formats, filters, etc, but I wasnt so lucky.

I’m worried about something, I don’t want end up doing the same I do everytime I start coding in a new language/platform. I always start wrapping the things I need, and keep doing it till I have been doing technical stuff so long that I get bored, instead of creating the application I create the tools, and the application never comes… It is a death spiral where I always fell. In game development there is a saying: “those who code game engines never code a game”, and it is true, as soon as you start coding your own engine you are always chasing a new feature instead of trying to create something with the features you already have.

So my plan now is to create the ultrabasic needs, which will be to have textures, shaders, RTs, maybe meshes (something simple) and a tiny graph editor, but I will talk about this when the time comes.

Ok, so I made a PostFX shader, I added the function to upload a texture to a shader and for my first test I rendered the same scene but reversing the colors and it worked.

gl_FragColor = vec4(1,1,1,1) – texture2D(texture,gl_TexCoord[0].st);

I was thinking of leaving that as an example of PostFX but that was too simple so I thought about something more complex.

Where you are coding a PostFX shader you read the pixel you are going to write from a texture, then you apply a function using the color of the pixel as the input and the output is returned from the pixel shader to be written in the same position of that pixel.

But sometime ago I realized that when it comes to PostFX you usually want to read not only that pixel, also the average of the pixels around that one, or in other words, you want to blur the pixel. To do that there are several options, one is to read all the pixels around your pixel from the texture and calculate the average, but for prototyping that is tedious, other solution is to blur the texture before passing it to the shader, but then if you want the original one you need two textures.

So I found a trick, to use the Mipmaps. Textures have mipmaps which are versions of the same texture but in lower resolution, and OpenGL allows you to create on-the-fly mipmaps of  a RenderTexture. It is kind of slow because it has to calcule all the mipmaps piramid even when you only want two or three levels, but the good thing is that it is only one line of code, so I added the option to the RenderTexture class.

I have used this trick in the past to create lens effect, motion blur, depth of field, etc. Obviously the results are not perfect in comparison with a good gaussian blur, but it does the task.

For this example I read the channels of the pixel separatly, with a little offset in X and I also add some blur to the red and blue component. Here are the results:

And here is the source code: hackpact day 3

For today I expected to clean a little bit more my code, it is growing fast from the skeleton I started and I don’t want to have to refactorice everything in a couple of days, but damn, I always end up wrapping stuff instead of using it so for this project I will focus on having the essential features.

First I wanted to add a simple wrapper to RenderToTexture features of OpenGL. The reason is that once you  have shaders and RenderToTexture you can achieve lots of cool algorithms, specially the postFX algorithms.

To do PostFX you want to be able to apply an algorithm to every pixel on the screen after you have rendered the scene, but OpenGL was thought more to render primitives on the screen (lines, triangles, and so). Then the old way to do PostFX to the final image was to download from the Video Memory the frame to the RAM, then using the CPU apply the algorithm, then upload it again, but that was far too slow and CPU consuming (and using Python thats MADNESS!!)

The cool way is using shaders and RenderToTexture combined. How? Well, Shaders allow to execute an algorithm for every pixel of the primitives we draw, not enough, because we want to read the previous pixel, apply the algorithm and store it in the same place. So the trick goes like this, we render the whole scene in a texture, then we render that texture in a quad filling the whole screen but with a shader with our PostFX algorithm activated.

So the shader is going to be executed for every pixel on the screen, and we can have the pixel in that position as an input to the shader. The result will be store it in the same pixel. Easy!

We can achieve nice effects like color balance, combine different images, blurs, edge detecting, etc.

So let’s start wrap a Render To Texture in a class, the interface should be simple, something like this:

• constructor where you choose Width, Height, Depth (and in my case some other flags)
• enable which redirects the rendering to our texture instead of the screen
• disable to restore the rendering to the screen
• bind to bind the resulting texture
• render to render the resulting texture as a quad to the screen

So after some work I managed to have everything working. Some interesting thoughts about it:

• Working with OpenGL in Python is easy, even some old OpenGL API functions have been wrapped in a more helpful way, for instance glGenFramebuffers in C++ you have to pass the reference to the variable where you want to store the Id of the FBO, but in Python it just returns it, thats nice.
• I had some problems when it comes to passing to OpenGL memory address, for instance if you create a texture and you want to initialize it with something. I don’t know how to do it, I have seen examples using Numpy but I just skipped that part.
• OpenGL usually never crashes, but those functions that expect a pointer… you better take care of them, because if you miss the address they will crass the application, no Python exception will catch that.

The best way to test if your Render To Texture works fine is to render the whole scene to a low resolution texture and show it to a full screen quad. You should get some pixelated results (if you disable filtering), here is my proof:

Once I have the Render To Texture results I could create a PostFX shader but I will do that tomorrow. For today I better play a little bit more with the scene I’d got from Bastiaan adding some input and cool effects like changing the size of the cubes depending on the distance to the light source.

If you want to test the application (I won’t suggest it in the early stages because there is not too much interesting stuff unless you want to know more about Python and OpenGL) here are the keys:

• 1 to 4: enables different effects (blend, rotate cubes, sphere shaped, edges and RenderToTexture)
• Space and Backspace: to move the camera back and forth
• Keypad + and – reduce the dimensions of the voxel
• ESC to close it

It is nice to use pygame underneath because it is a wrapper of SDL and I have been using SDL for years so most of the features of SDL are exposed with similar names and interface but in python. For instance, I wanted to use the time increment to move the objects instead of the frame number, this is important to have a constant movement, otherwise it moves faster when the scene is emptier (when the framerate is higher).

I also added a small bar at the botton to show the framerate, I don’t know how to render text (I probably will need to use another module…) so I just draw a line from left to right, where right means 60fps and left means 0fps.

I also made some small optimizations to the Bastiaan code, nothing to be ashamed for, just ensure all the faces of the cubes were clockwise so I can enable back face culling.

If you want to test it the files are zipped here: hackpact day 2

Real-time graphics is a field ruled by C++, not by imposition, just because the idiosyncracy of the task. We need to do a lot of things and we need them to be done as fast as possible, otherwise we have a drop of the framerate and we loose the feeling that the world is moving.

So C++ works perfect in that field, in the field of real-time standalone graphics applications. But sometimes we want more, we want other things, like flexibility, like on-the-fly coding, like portability, and then we start looking C++ like and old man looks at his wife, yeah, she was adorable, but a thousand years ago.

I like coding graphics, it reminds me drawing and I like drawing. Somehow it is the same in my mind, but with the difference that coding I can make iterative things a lot faster and easier. I am part of a small set of people in the world who know how to draw and how to code graphics, and I want to take advantage of that. I want to explore both fields till the point where they merge in a new form of expression.

So today I’m gonna park my C++ compiler and start with something new, I need a language powerful (but not too much), where I can use my knowledge of graphics but with the flexibility of a playground, a sandbox, where I can add and remove in a simple way, within a keystroke. The idea? To create an application where I can code graphics meanwhile they are renderer, similiar to fluxus but using OpenGL paradigm.

And the awarded platform is… Python!. I know enough of this language to know it fits my needs, there are modules for almost everything and it is made to be confortable to code, but the setting up is going to be a pain in the ass.

So my first task today (and the one I’m going to document) is how to render some stuff in OpenGL using Python,

The setting up

First I have to download a thousand of packages:

• Python compiler (I choose 2.6 because 3.x version still don’t have a good support for most of the libraries)
• PyOpenGL all the binds needed to use OpenGL
• OpenGLContext which is a library wrapping the common stuff in all OpenGL applications, helpful, but this have depencencies too…
• PyDispatcher a library to send messages between elements
• PIL to load images
• Numpy to numerical actions along arrays (important to process meshes… I guess)
• PyGame more or less like OpenGLContext but with more stuff about games
• PyVRML97 this sounds kind of cheesy because who uses VRML now… but OpenGLContext asks for it

To install this libraries is easy, you just need to unzip them and execute from the command line “python setup.py install” inside the folder, and they do the rest.

I needed to test that everything was working so I end up in the website of Bastiaan Zapf: Complete Python OpenGL examples. Lot’s of useful examples of python mixed with OpenGL and with shaders!.

It took me some time to have all the libraries used by him, but it is a nice way to start, to take other person’s work instead of starting from scratch (and giving full credit!).

The first test

But after reading some of the examples propose by Bastiaan I better backup up a little, I mean, I need the basics of the aplication flow inside Python, things like creating a window, reading the keyboard input, rendering the frame, so I will follow the pygame tutorials to see what they suggest… No, too basic. They can be useful but I don’t want to waste time reading them. I just grabbed information to create and destroy the window and to check the keyboard input, for now I don’t need anything else.

I have an idea to start, I’m going to pack most of the import stuff I need to code graphics in some classes, stuff like Shaders and RenderToTexture, that is something common when you start coding in OpenGL, the API is boring and confusing, so the best thing to do is to encapsulate the API in simple calls closer to your mind paradign. I have the code in Bastian examples to compile a shader, so time to take it.

Here is my first “module”, just a small source file that will help me as a guide to future modules, I put there the shader compiler and how to create a box in a call list: GLTools

And then I clean up the Bastiaan file to use my tools and add some small features: Hackpact_dat1

And here are the results:

Cubes and blending

Main changes are to render two cubes instead of one (one in wireframe and one normal) and to use additive blending. I like when you overdraw the scene, it looks “hackerish”.

Conclusion

Hey, programming in python with OpenGL looks easier than what I thought. I don’t have to say the types and the API is totally exposed so I can do most of the regular stuff without thinking too much. The performance is good enough and it blends perfectly with Python.

Tomorrow, more test

My friend Graham is a good link to the productivity path of the sourcecoder. He always surprise me with his good use of the spare time, and he make me remember the golden years when I enjoyed spending hours trying to figure out the secrets of the ancestral art of developing for your own pleassure.

So sometimes we talk about metadevelopment (how to develop your skills to develop better). He told me about meetings he arranged back in the days with other people in the same situation, just to talk about their own projects, in an effort to push each other in the hard way to complete their ideas and to prevent them of parking the projects when things get hard.

And yesterday he told me about a Hackpact arranged by a friend. As far as I understood the Hackpact is a pact between a bunch of people with ideas where everybody undertakes to develop and document a small idea every single day for a month.

Sounds a lot of work to me, but hey, that’s what I need now, a virtual pact with some foreign people I don’t know for a greater purpose, in my case, the glorious exploration of the real-time rendering using scripting languages.

So here is my pact, but don’t blame me if I only last for a week, it was enough effort to recover this blog!