How to Texture-Map Triangles in a Canvas
Some who are curious as to how it was possible to achieve texture-mapping in an HTML5 canvas probably already took a peek at the code in the Wii Opera SDK only to see that it has been optimized into obfuscation. Thus, I’ve decided to write an article to explain the technique, starting with the basics.
The triangle definition
The triangles are defined in the same manner as they are in the flat shading part of the SDK and are fully compatible with any meshes (discarding color information) until the final drawing step. For developers who wish to hard code the triangle coordinates, they are defined in a standard computer floating point coordinate system where X is positive rightward; Y is positive downward; and Z is positive outward. The triangles have nine numerical values, which are three coordinate sets, counterclockwise. Normally there are three final values for the RGB flat shading colors, but these get replaced in the final step by an image object reference. Note that this image object can be of any size, but Opera’s speed decreases as the image size increases, and there is no need, anyways, for large textures in an Internet browser environment.
The texturing method definition
The values passed to the method will be the 3D to 2D translated coordinates, so there will be six numerical values instead of nine. Thus, the method will be called as follows:
Draw.textureTriangle(x1,y1, x2,y2, x3,y3, image);
Triangle Drawing Premise
Since the canvas object lacks 3D image transformation, the texels have to be generated in another manner. For transformations, the following are the ones Opera implements in the standard canvas context:
1. Rotation (angle)
2. Scaling (sx, sy)
3. Translating (x, y)
For image drawing, the canvas allows rectangular blocks, nothing skewed. But by first rotating the canvas, the entire source image can be rotated like the ships in HullBreach Online. Well, this is fine if the triangles are all right triangles with the right angle at the first vertex, but isosceles and scalene triangles gain no help from it. Here’s where the trickery comes into play…
Triangle Drawing Setup
The first step is to make the math as simple as possible. This means creating axis-aligned edges, zeroed coordinates, normalized sides, or anything that will help with speed and ease of the math.
The easiest way to start this is to first translate the canvas’ coordinate system in the direction of (x1, y1). Then all the coordinates of the triangle can be shifted (-x1, -x1), making that vertex now (0,0). That causes math to be much simpler because it reduces some of the coordinate calculations to zeroes. Then by finding the arctangent of the first side’s direction, the rotation amount is discovered. In a similar premise to the previous translation, rotate the triangle in the direction of the newly-discovered angle, zeroing y2 to make x2-x1 (or more simply: just x2 due to the zeroing) the u vector (the maximum x width). This also makes y3-y1 (or just y3 due to zeroing) the v vector (the maximum y height). To preserve the angle of the triangle’s first side, rotate the canvas in the opposite direction to the angle.
The triangle is now ready to draw!
Drawing the Triangle
The triangle will be drawn using scaled horizontal scanlines from the source image, using the drawImage method of the canvas object. At the base of the triangle, a copy will be the entire width-1 (to prevent an overflow) of the source image, and the draw will be the entire u vector width. As the loop moves up the v vector, the copy width approaches 0, and the draw width scales to 0. To create the skewing of the triangle’s third edge, the destination x will be a traversal averaging of x1 and x3, where it is 100% x1 at the beginning and 100% x3 at the end. Scaling the width of the copy and the drawing takes care of the second edge, so no further calculations are needed.
Conclusions
I’ve tested this texture-mapping method with several meshes, and the speed seems fairly constant, no matter how many triangles are in the mesh, as long as those meshes are roughly the same size. The biggest speed concern is size. As the triangle sizes increase, speed drops considerably. Thus, for textured meshes of interactive or passable speeds, they need to be kept fairly small.
I’m curious to see what developers are able to make for the Internet Channel - and even on the PC - using this. In the meantime, I have several ideas I wish to implement.
Tags: 3d, canvas, mapping, opera, sdk, texture, triangles, wii
April 30th, 2008 at 6:19 pm
Nice explanation. Now that you have added image to the Draw and 3D2 libraries, I can start using 3D2 to develop a game with meshes as environment as well as objects. Question, is there a performance hit to having many meshes out of the view, that is, when rotating in a crowded scene, does it automatically eliminate triangles from calculation, which are invisible because they are behind the viewer?
April 30th, 2008 at 6:31 pm
Actually I see the 3d2.js is not changed yet, so that image info can be passed to drawScene from GetTranslation… will you be doing that? If not, I guess I can use special RGB combos to indicate certain images, then cycle through the triangles as you have done in the texturemap.html
April 30th, 2008 at 7:28 pm
3d2.js auto-clips triangles that are behind the viewer when using GetTranslation. What I’m thinking about doing to better integrate the texture-mapping into scene drawing in a future update is to allow passing of a reference to an image array for textures, then one of the colors (Red, for instance) for the mesh could be the reference to that texture, serving a dual purpose like so:
Flat shaded:
Triangle = new Array(x1,y1, x2,y2, x3,y3, r,g,b);
Textured:
Triangle = new Array(x1,y1, x2,y2, x3,y3, image,dummy,dummy);
I hope that makes sense. I’m just waiting for Opera to support skewing of images in a canvas like FireFox does so that I can draw textured triangles in a single pass instead of by scanline. Imagine the speed increase! One could potentially create games with full texture-mapped scenes at decent framerates instead of being limited to just low-triangle objects at the moment.
May 9th, 2008 at 12:12 pm
One nice thing about the canvas object is that is automatically clips any drawing that is outside the scene, thus speeding up drawing that falls outside the bounds. From what I’ve seen in simple tests, the coordinates of pixels inside a canvas appear to have the bounds of signed 16-bit values [-32768, 32767] but are actually single-precision floating point values. The decimal values are preserved when plotting pixels with the canvas object automatically anti-aliasing that pixel to cross over the appropriate visible pixels.
I haven’t had a chance to update the texture method to have a TextureScene() equivalent of DrawScene() due to several other projects, but that will be coming very soon. Expect the R of the RGB to reference a texture number from an image array and the G and B to be reserved values for the moment.
May 12th, 2008 at 1:24 pm
I assume the TextureScene will check if R is numeric and use flat-shading if so, so image textures can be used only on the necessary objects? Will it make sense for the G and B to be used to enable darkening of the image based on Shade reducing them from initial 255?
May 12th, 2008 at 3:35 pm
R would have to be a non-negative integer. I think I’ll make G be the texel scaling factor, which could cause faster rendering time at cost of image quality. If you can think of what B should be, I’ll go ahead and start integrating it all into a new set of functionality.
May 15th, 2008 at 7:43 pm
I think B should start as 255, and be multiplied by cos as usual in Shade or Shadow, so it can be used as a darkening factor when rendering the image… I just don’t know how you would use that information in TextureScene to dynamically darken the image… other than having several versions of the image, with different B values appended to the name, for levels of darkness.
May 16th, 2008 at 11:35 pm
That idea of the B as sort of a brightness channel is good. It could be applied to a triangle by treating 0 as the null value with a full bounds of [-255, 255]. A negative number will draw black pixels over the top with alpha of -[0,-255]. A positive number will draw white pixels over the top with alpha of +[0,255]. How’s that idea? It wouldn’t really slow down the drawing too much, and the value of 0 wouldn’t slow it down at all.
May 21st, 2008 at 7:53 pm
That sounds good! If it doesn’t complicate the shading algorithm. I was thinking to keep it in line with the flat colors, where the provided image is the brightest it can get, and it darkens to black from there. It could be nice to brighten images close to the light source, but would the flat colors brighten also? (I plan to use flat colors in combination with images, where appropriate.)
May 24th, 2008 at 7:06 pm
Here you go TK: http://www.wiioperasdk.com/texturemap.html . That’s a demo of realtime shading of textured triangles. For whatever reason, it’s flakey in FireFox, but it works great in Opera. Apparently FireFox doesn’t like the usage of image arrays passed to objects. I’ll find a fix for it.
The triangle values are the following:
- 3D -
x1,y1,z1, x2,y2,z2, x3,y3,z3, textureIndex, texelSize, brightness
- 2D -
x1,y1, x2,y2, x3,y3, textureIndex, texelSize, brightness
The textureIndex refers to where in the texture array that triangle reads. The texelSize determines how wide each scanline is with larger values drawing faster at the cost of image quality. The brightness is a value from -255 to 255, which determines if an image is darkened or lightened when drawing.
I’ll write up a tutorial for the changes to the SDK, but for now you can take a peek at that link’s source code to get an idea about the new functionality that’s been added.
June 2nd, 2008 at 8:11 pm
Thanks! I assume you have updated the 3d2.js and draw.js used in texturemap.html to include the Tangent128 accelerations?
I was thinking of not needing separate functions for texture and color, rather allowing the mesh definition to include some texture triangles and some flat ones, and all the functions handle either appropriately. But if the texture rendering is fast enough, maybe I can just go with all textures… in which case we can remove the flat-color code to optimize.
June 4th, 2008 at 8:29 pm
I have some ideas in mind for version 3.x of the SDK. One such idea is to redefine meshes (and provide a converter from the old format, of course). Meshes could contain something like this:
x1,y1,z1, x2,y2,z2, x3,y3,z3, texture, brightness, r,g,b
x1,y1, x2,y2, x3,y3, texture, brightness, r,g,b
A texture value of -1 could flag the function to know that the triangle is flat-shaded instead of textured. The speed decrease would be minimal to do a simple if-then check, but it could potentially increase the execution speed with less code for Opera to scan and parse.