What a mess! The next big step to take for 3D graphics in VisualWorks is to get some serious models in to the system, with bones, animations, facial structures and animations, UVW mapping, high polygon-count -> low triangle count with bump mapping, etc. It turns out this is a huge black hole of technology.

Premise 1: Making a 3d modeler is a lot of work, not because of the rendering and interactive capabilities - both of which you'd need for any serious 3d program/game - but because of the transformative properties of such an application. You also have to consider the kind of modeler you want to make. The simplest is polygonal modeling, but this is incredible difficult to use to make organic looking shapes. Next is NURBS which is used by the big guns, Maya and 3ds Max. Splines and patches are the poor mans version of NURBS. Next, Primitives which is a set of geometrics such as balls, cyilnders, cubes that can be used and parameterised as the building blocks of a model. Finally, there is the newer techniques of sculpt modeling where you use things like subdivision surfaces or voxels to build shapes from blobs.

Premise 2: Importing from a proprietary format or open format is less work than building a whole modeler, so let's just do that kthx bai. Well, interesting in theory but it turns out in practice there are a few stumbling blocks here. First off, most of the formats you can use do not include any bone structures or animation. If they do include animation often it is a single key-frame timeline without any markers to indicate start/stop points of an animation, let alone evolution between animation modes.

The 'serious' format of the business right now is still .fbx which is Autodesk's format supported by Maya and 3ds and it is the main import format in to Microsoft's XNA. This is the most 'standard' of the formats because they provide a free SDK written in C++ to import their .fbx files in to memory for you to then render. The format is proprietary and as we well know, integrating with a C++ library from VisualWorks is a real pita. Despite this, fbx includes animation data, bone structures, just about everything you might ever need to render something properly. Yet, Microsoft built pipeline tools to convert the .fbx format in to their own format - the SDK contains a HUGE swath of code to render a scene, manipulate cameras, etc, all of which is redundant and out dated when you build a custom rendering pipeline such as deferred shading and SSAO.

Premise Each format of course is dictated by the kind of modeling technique used - as described above, are you going to get lists of triangles, NURBS, patches or primitives? What do you do with all that once you've got it. One of the advantages of the .fbx SDK is that it comes with an API to tesselate the model in to triangles - I expect this is what Microsoft does in their pipeline. But even here we're making too many assumptions about the end result dataset. What if we want more triangles? What if want to bump map?Too many what ifs.

3: It's what the big guys do, so it's probably what we should do. Cry Engine 3, one of the more famous 3d engines around has ways of importing content just like Microsoft XNA. But, it becomes apparent very quickly that all but the most simplest of data comes from the 3d modeler in to Cry Engine. Terrain is their game, they build roads, trees, oceans, mountains. They also do facial animations, body animations, bone structuring, sound tagging.. all kinds of stuff that cannot be done inside of 3ds Max or Maya. So essentially, even when you're using the biggest baddest 3d engine out there, you're molding the clay, then moving over to their engine to do the heavy lifting.

Conclusion: Everything is one giant mess. Even if you go with 3ds Max or Maya, or a free variant such as Wings 3d or a cheap variant such as Milkshape 3D, you're still only going to get as much data out of it as you can invest in to it - which is to say, usually not very much. If we go back to Premise #1 we posit that it's hard to make a 3d modeler. It's probably not as hard as everything else you have to do after you've made your model. Cry Engine 3 lets you build the world while you're in it; sound familiar? very smalltalk-ish isn't it. So why not have the whole tool chain from go-to-woe all in the one engine? Then, rendering in Maya is never going to look different to how it looks in the final rendering engine with sun-moon cycles, retinal brightness adjustment, material and shader integration, mood lighting, sound integration (yes, sounds do affect the way you see things), etc.

What this has led me to conclude is that the main reason to support 3ds Max or Maya is to import talent - any one who is familiar with these tools can build you 3d models very easily. Unfortunately for you, the models will often be boneless, animation-less, facial animation-less, jointless, muscleless and be stacked with all kinds of custom shader options, unique primitives you don't support, etc.

What's the alternative? Make your own 3d modeler as part of your 3d engine. The downfall here is that you have to train up the talent to use your software - however, your entire content pipeline can be fully integrated between the software and the modelers as well as the world/terrain editor. There is another hidden pitfall too - your assets are less portable if they're only stored in your own proprietary format. I'd bet money that Duke Nukem Forever hit this problem when they were porting between engines.

So my next step now is to take the dive, start making a 3d modeler / world / terrain editor all in one (since they are very similar in nature) which will further allow me to build a geometry rendering / deferred shading renderer with SSAO for high quality rendering, as well as day-night cycle. Two other engines that aren't "big guns" that I take inspiration from are Quel Solaar and Leadworks. If nothing else, it'll educate me more and I should have fun trying.