Over the weekend I got adventurous with some code I'd written for OpenGL that could recognize and map between different color spaces, or palettes as VisualWorks calls them.

This all stems back to the Image object and its subclasses, DepthXXImage. They have a palette and some raw binary data that the VM knows about. The bitblt operations in the VM know how to copy the bits around and given that the two palettes are the same, copying can be very fast. That's all great, except that when we want to interface with something other than the VMs bitblt, we suddenly have a problem.

Both CairoGraphics and OpenGL need to know the kind of image binary data they're dealing with. CairoGraphics is pretty mundane in this respect - it has two primary formats, RGB24 and ARGB32 so copying the binary data from one Image format in to that image format should be pretty easy - however, the code to do it is very ugly - and if our image is already in the right format, we can't easily tell and avoid the copy.

OpenGL, on the other hand, can take just about any memory format under the sun. In fact, it's so smart, you can specify your data is ARGB and tell it that it's in integer format split up by groups of 8 bits in -reverse- order. Very very flexible system. What's more, most of those operations actually then happen in the hardware, costing us no time at all no matter what strange format we've got our image in - so long as we can tell what kind of format we're in.

So I had some code in the OpenGL package which knew a couple of commonly used formats, ARGB, ABGR, RGBA, BGRA, so on and so forth.. but this didn't exactly help me convert between Graphics.Image and CairoGraphics.ImageSurface, so I made a new package called Graphics-Images-FIxedPalettes which adds in a whole slew of image binary formats that you commonly find in graphics packages.

The new palettes provided are:

• 1bit grayscale: A_1
• 8bit grayscale: A_8
• 8 bit 2/3/3: RGB_332 BGR_233
• 16 bit 4/4/4/4: ABGR_4444 ARGB_4444 BGRA_4444 RGBA_4444
  
• 16 bit 5/5/5: RGB_555 BGR_555
  
• 16 bit 5/6/5: RGB_565 BGR_565
  
• 16 bit 5/5/5/1: ABGR_1555 ARGB_1555 BGRA_5551 RGBA_5551
  
• 24 bit 8/8/8: RGB_888 BGR_888
  
• 32 bit 8/8/8/8: RGBA_8888 BGRA_8888 ARGB_8888 ABGR_8888
  
• 32 bit 10/10/10/2: RGBA_1010102 BGRA_1010102 ARGB_2101010 ABGR_2101010
  

The selector format deserves some explanation. The selectors are designated by Most-Significant-Bit to Least-Significant-Bit. For example, ARGB_8888 is ordered from MSB channel A (alpha) to the LSB channel B (blue). When traversing image data on *little endian* machines, the data is stored in memory from LSB to MSB, which means that the channel order in memory is from LSB (blue) to MSB (alpha).

Okay, so now we have this, what good is it? Well, it's now really easy to tell what kind of format an image is in - in fact, the palettes themselves often know what format they are (thanks to the constantNames mechanism that Palette provides).

Now it became easy to implement #asImage on CairoGraphics.ImageSurface to transform its data in to a Smalltalk Image object - and vice version, you can send #cairoSurface to an Image and get back a CairoGraphics.ImageSurface.

It's now that simple to reuse image assets already in the Smalltalk image in Cairo - but why stop there. We can also now do the same in OpenGL, except that it's even more efficient. Most of the time, the images in Smalltalk are not in a format that CairoGraphics recognizes.. however, OpenGL doesn't have that problem, it'll take just about any format under the sun. Now that we know what format an image is in, we can turn that in to OpenGL constants, such as GL_RGBA with GL_INT_8_8_8_8_REV or whatever the situation might be.

What does all this mean for OpenGL? It means we can reuse existing Smalltalk Image assets as textures in OpenGL as well. It also means that for both CairoGraphics and OpenGL our existing ImageReader classes can continue to be used (if so desired, though they may not always do what you want).

Alpha channel information is now recognized when you're using Depth32Images and if you convert from a <32 bit image to an ARGB (or similar) palette, it will actually give you back a Depth32Image instead of a Depth24Image.

In a round about way, I still wasn't done - I've been drawing fonts in OpenGL for a while now but if you're not on the Windows platform, you can kiss your anti-aliasing good bye.. or in otherwords, the font rendering I'd made looked incredibly crap.

In steps CairoGraphics. I've got a package called OpenGL-CairoGraphics which allows you to use CairoGraphics.ImageSurface's as textures and I made a demo of how to do that called CairoGraphicsTextures in the OpenGL-Examples package about two months ago. I was feeling pretty good about it because OpenGL can read the data directly and again, like Images now, didn't have to transform it in memory - the hardware can do it for us (or not at all if that's the optimal render pipeline format).

Now I've added a new piece of functionality to OpenGL-CairoGraphics - OpenGLCairoFont. Instead of creating a font 'natively' (which really only works well on Windows), you can create it from fonts provided by CairoGraphics. The neat thing here is that Cairo is already linked in to the platforms true type font system with anti-aliasing image surfaces, so you can get very high quality font renderings.

The class will render each of the visible characters on to a single OpenGL texture and then when it draws text, it creates quads for each character (which is very cheap) and sets the texture on the quads offset by the position of the character we want to display (which is also very cheap). All in all, this means that rendering text in OpenGL this way is -incredibly fast-. And by fast I mean, sometimes a lot faster than a regular font rendering engine! I noticed that IO Language guys are doing something similar with their fonts and they also noted the performance gains. Thumbs up.

One more thing before I wrap up this CairoGraphics and OpenGL blog post - Drawing in 2D. No problem for CairoGraphics ;) .. but in OpenGL drawing in 2D can be a real nightmare. Once I could draw my fonts nicely I discovered an interesting problem - they were huge, hugely huge.. i had to majorly scale back my matrix to get the text to display like you'd expect.

This caused me an interesting new problem... where is the edge of the screen? Well, the obvious answer to this problem is to swap the projection matrix in to Ortho mode where drawing is done parallel to the camera which is aligned to the viewport of the window. However, you may quickly realize that mixing 3d and 2d together can be a real pain if you have to constantly swap out of perspective projection and into ortho projection.

I added a new set of methods to OpenGLContext which let you convert window coordinates in to object-space coordinates and back again. I then use this information to provide a new API called #transformToWindowBounds. This method will scale the z=0 plane such that each object-space coordinate matches 1:1 with window coordinates. The matrix is transformed so that 0, 0, 0 is at the window coordinate of bottom-left. This lets you simply modify the matrix with this one call to swap to 2d mode without actually leaving 3d mode.. you can even render depthed stuff in this mode, like a sphere or something.. and it'll work.

I made a new demo for this called Draw2D_Perspective and it's also in OpenGL-Examples.

As of this writing, the versions I'm using are:

• LibCairo 1.6.x (as in, the dll/so/dylib)
• CairoGraphics 80 mls + Image surfaces.3
• Geometry-Vector 13
• Graphics-Images-FixedPalettes 4
• OpenGL 2.97
• OpenGL-CairoGraphics 4
• OpenGL-Examples 65
• OpenGL-Lessons 2
• OpenGL-Linux 13
• OpenGL-MacOSX 18
• OpenGL-VisualParts 2
• OpenGL-Windows 13

Happy CairoGraphicsing and OpenGLing!