Canvas is a 2D drawing API recently added to HTML and supported by most browsers (even Internet Explorer 9 beta). Canvas allows you to draw anything you want directly in the webbrowser without the use of plugins like Flash or Java. With a deceptively simple API, Canvas has the ability to revolutionize how we build web applications for all devices, not just desktops. This session is a three hour workshop that will deep dive into Canvas, starting with the basics and progressing into real world applications. Finally we will take a look at some experimental edges of Canvas, such as webcam and audio visualization.
This is a technical talk. Basic knowledge of JavaScript programming is required.
Hi. Welcome to the HTML Canvas Deep Dive. Today we will dive very deep into the Canvas API and learn how to make both useful and graphically intense effects using pure web technology, no plugins required. My name is Josh Marinacci and I'm joined by Robert Burdick. We are from the webOS evangelism team at HP. We are teaching this class on HTML 5 Canvas because we built webOS on web standards and really feel that it's the way most apps will be built in the future. I'm also happy to say that our recently released HP TouchPad (hold it up) was reviewed by the Sencha team and they said it has the fastest mobile Canvas implementation they have seen, hands down. So we really do practice what we preach.
This class will have three lecture sections where we will learn about the Canvas API, how to do animation and pixel effects, and some real world tools to help you. In between the lecture sections we have hands on lessons for you to walk through and build your own canvas apps. During these parts Robert and I will walk around to help you and answer questions. In terms of skill you only need to know some basic javascript and HTML. All you need on your computer is a copy of Chrome or Safari and your favorite text editor. Canvas is very easy to work with. No IDEs required.
So lets get started!
Canvas is a 2D drawing API. Essentially the web browser gives you a rectanglar area on the screen that you can draw into. You can draw lines, shapes, images, text; pretty much anything you want. Canvas was originally created by Apple for their Dashboard widgets, but it has since been adopted by every major browser vendor and is now part of the HTML 5 spec. Here's a quick example of what some Canvas code looks like:
<html>
<body>
<canvas width="800" height="600" id="canvas"></canvas>
<script>
var canvas = document.getElementById('canvas');
var c = canvas.getContext('2d');
c.fillStyle = "red";
c.fillRect(100,100,400,300);
</script>
</body>
</html>
Now it's important to understand that canvas is for drawing pixels. It doesn't have shapes or vectors. There are no objects that you can attach event handlers to. It just draws pixels to the screen. As we shall see this is both a strength and a weakness.
SVG: SVG is a vector API that draws shapes. Each shape has an object that you can attach event handlers to. If you zoom in the shape stays smooth, where as the canvas would start to become pixelated.
CSS: CSS is really about styling DOM elements. Since there are no DOM objects for things you draw in the canvas you can't use CSS to style it. CSS will only affect the rectanglar area of the canvas itself, so you can set a border and background color, but that's it.
DOM animation: The DOM, or Document Object Model, defines an object for everything on the screen. DOM animation, either by using CSS or javascript to move objects around, can be smoother in some cases than doing it with canvas, but it depends on your browser implementation.
So when should you use Canvas over SVG, CSS or DOM elements? Well, Canvas is lower level than those others so you can have more control over the drawing and use less memory, but at the cost of having to write more code. Use SVG when you have existing shapes that you want to render to the screen, like a map that came out of Adobe Illustrator. Use CSS or DOM animation when you have large static areas that you wish to animate, or if you want to use 3D transforms. For charts, graphs, dynamic diagrams, and of course video games; Canvas is a great choice. And later on we will discuss a few libraries to let you do the more vector / object oriented stuff using Canvas.
Before we go any further I want to clarify that when I'm talking about Canvas I mean the 2D API. There is also a 3D API in the works called WebGL. I'm not going to cover that here because it is still being developed and the browser support is rather poor. Also, it's essentially OpenGL from JavaScript, making it lower level than Canvas and much harder to use. When WebGL becomes more mature we will revisit it.
And lastly, before we dive into working with Canvas, let's talk about where you can use it? Fortunately Canvas is now a stable API and most modern browsers support it to some extent. Even Internet Explorer supports it starting with IE 9, and their implementation is very good.
Safari >= 3 Chrome >= 10 Opera >= 9 FireFox >=4 Internet Explorer >=9
On the mobile side most smartphone platforms support it because most of them are based on WebKit, which has long had good support. I know for sure that webOS, iOS, and Android support it. I believe BlackBerry does, at least on the PlayBook. Windows Phone 7 does not, but it may come in a future update.
webOS iOS Android BlackBerry (PlayBook and OS 6.0+) NOT WP7
Now, not every mobile device has very complete or fast support for canvas, so we'll look at how to optimize our code for mobile devices later in the performance section of this session.
As I said before, Canvas is a simple 2D API. If you've done any coding work with Flash or Java 2D it should seem pretty familar. You get a reference to a graphics context, set some properties like the current fill or stroke color, then draw some shapes. Here's a couple of examples.
c.fillStyle = "red";
c.fillRect(100,100,400,300);
In this example we set the current color to red and draw a rectangle. Here's another one.
c.fillStyle = "#ccddff";
c.beginPath();
c.moveTo(100,100);
c.lineTo(400,50);
c.lineTo(400,300);
c.closePath();
c.fill();
c.strokeStyle = "rgb(0,128,0)";
c.lineWidth = 5.0;
c.stroke();
In this example we set the current fill color, create a path, then fill and stroke it. Note that the context keeps track of the fill color and the stroke color separately. Also notice the different forms of specifying colors. fillStyle and strokeStyle can be any valid CSS color notation like hex, names, or rgb() functions.
A quick word on coordinate systems. Canvas has the origin in the upper left corner with the y axis going down. This is traditional for computer graphics, but if you want a different origin you can do that with transforms, which we will cover later. Another important thing is that the Canvas spec defines coordinates at the upper left corner of a pixel. This means that if you draw a one pixel wide vertical line starting at 5,0 then it will actually span half of the adjacent pixels (4.5 to 5.5). To address this offset your x coordinate by 0.5. Then it will span 0.5 to the left and right of 5.5, giving you a line that goes from 5.0 to 6.0. Alternately, you could use an even width line, such as 2 or 4.
Canvas can fill shapes with things other than colors, BTW. We can also fill with an image or gradient.
var grad = c.createLinearGradient(0,0,200,200);
grad.addColorStop(0, "white");
grad.addColorStop(1, "black");
c.fillStyle = grad;
c.fillRect(0,0,400,300);
var grad = c.createLinearGradient(0,0,200,200);
grad.addColorStop(0, "white");
grad.addColorStop(1, "black");
c.fillStyle = grad;
c.fillRect(100,100,400,300);
So if you get into a case where you think you are filling a shape with a gradient but only see a single color, it might be because your coordinates are off.
You can also fill shapes with images by defining a pattern. You can control how the pattern is repeated the same as you would with background images in CSS.
var img = new Image();
img.onload = function() {
var canvas = document.getElementById('can');
var c = canvas.getContext('2d');
var pat = c.createPattern(img,'repeat');
c.fillStyle = pat;
c.fillRect(10,10,80,80);
var canvas2 = document.getElementById('can2');
var c2 = canvas2.getContext('2d');
var pat2 = c2.createPattern(img,'repeat-y');
c2.fillStyle = pat2;
c2.fillRect(10,10,80,80);
}
img.src = 'smile.png';
Note that this filling with an image texture only works if the image has already been loaded, so up here I'm only doing the drawing from the image onload callback.
In addition to these different fills we can control the opacity when we draw. Suppose we wanted to draw a red rectangle and then a white but 50% translucent rectangle on top of it. Here's how we would do it:
c.fillStyle = "red";
c.fillRect(100,100,400,300);
c.globalAlpha = 0.5;
c.fillStyle = "white";
c.fillRect(100,100,400,300);
c.globalAlpha = 1.0;
This opacity setting works with all drawing operations. Be sure to set it back to 1.0 when you are done so that it won't affect later drawing.
Paths are shapes created by a bunch of straight or curved line segments. In Canvas you must first define a path with beginPath(), then you can fill it, stroke it, or use it as a clip. You define each line segment with fuctions like moveTo(), lineTo(), and bezierCurveTo(). This example draws a shape with a move to, followed by a bezier curve segment, then some lines. After creating the path it fills and strokes it with the current fill and stroke styles.
c.fillStyle = "red";
c.beginPath();
c.moveTo(10,100);
c.bezierCurveTo(20,200, 500,200, 500,100);
c.lineTo(500,300);
c.lineTo(10,300);
c.closePath();
c.fill();
c.lineWidth = 4;
c.strokeStyle = "black";
c.stroke();
In addition to using images as patterns you can simply draw them to the screen with drawImage.
var img = new Image();
img.onload = function() {
c.drawImage(img, 0,0);
c.drawImage(img,
20,20,30,30, //source coords
0,0,200,200//dest coords
);
c.drawImage(img,
40,20,20,20, //source coords
250,50,250,50//dest coords
);
}
img.src = 'smile.png';
As you can see there are several forms of drawImage. You can draw the image directly to the screen at normal scale, or you can stretch or slice it how you like. Slicing and stretching images can be very handy first special effects in games because image interpolation is often much faster than other ways of doing scaling.
c.fillStyle = "black";
c.font = "italic 96pt Arial ";
c.fillText("this is text", 50,300);
You can draw text with Canvas too. The font attribute is the same as its CSS equivalent, so you can set the style, size, and font family. Note that the fillText() function uses the y value for the baseline of the text, not the top. If you put your text at 0,0 then it will be drawn off the top of the screen.
So that's it for basic drawing. Let's stop there and do some exercises. You should already have a webbrowser and text editor installed. I recommend using Chrome because it has nice debugging tools, and jEdit because it's free and cross platform; but you can use the browser and editor of your choice. Note that since we are loading code from the local harddrive you may need to disable security in Chrome during development. The handout notes describe how to do this. The notes can be viewed here:
Hands On Lab 1: Bar and Pie ChartNow that we know basic drawing lets dig in a bit deeper. In your bar chart you drew the same rectangle over and over again just with different x and y coordinates. Rather than modifying those coordinates we could have used a translate function instead like this
for(var i=0; i<data.length; i++) {
c.translate(40+i*100, 460-dp*4);
var dp = data[i];
c.fillRect(0,0,50,dp*4);
}
Like many 2D APIs, Canvas has support for the standard translate, rotate, and scale transforms. This lets you draw shapes transformed around on the screen without having to calculate new points by hand. You can also combine transforms by calling them in order. For example, to draw a rectangle translated to the center and then rotated by 45 degrees you would do this:
c.fillStyle = "black";
c.translate(300,200);
c.rotate(Math.PI*2.0/8.0);
c.fillRect(0,0,100,100);
Each time you call translate, rotate, or scale it adds on to the previous transformation. Over time this could get confusing, of course. You could undo the transforms like this:
for(var i=0; i<data.length; i++) {
c.translate(40+i*100, 460-dp*4);
var dp = data[i];
c.fillRect(0,0,50,dp*4);
c.translate(-40-i*100, -450+dp*4);
}
but that's a lot of annoying code to write and if you forget to undo it just once then you could be screwed and spend hours looking through your code for that one bug. (not that I've ever done that, of course!) Instead we can use state saving.
The context object represents the current drawing state. The state includes the current transform, the fill and stroke color, the current font, and a few other variables. Canvas lets you save this state by pushing it onto a stack. After you save the state you can make modifications, then restore to the previous state. Canvas takes care of the book-keeping for you. Here's the previous example done using state saving. Notice that we don't have to do the untranslate step.
c.save();
c.fillStyle = "red";
c.translate(100,100);
c.fillRect(0,0,100,100);
c.restore();
c.fillStyle = "blue";
c.fillRect(0,0,100,100);
Sometimes you may want to draw just part of a shape. You can do this with the clip function. It takes the current shape and uses it as a mask for further drawing. This means that any drawing will only happen inside of the clip. Anything you draw outside of the clip will not be shown on screen. This can be useful for when you want to create a complex graphic by combining shapes, or when you want to update just a part of the screen for performance reasons. Here's an example where we draw a bunch of squares clipped by a circle:
c.strokeStyle = "black";
c.lineWidth = 10;
c.fillStyle = "red";
// draw rect the first time
c.fillRect(0,0,600,200);
c.save();
// create triangle path
c.beginPath();
c.moveTo(200,100);
c.lineTo(300,300);
c.lineTo(100,300);
c.closePath();
// stroke the triangle so we can see it
c.stroke();
// use triangle as clip, then fill rect in again with yellow
c.clip();
c.fillStyle = "yellow";
c.fillRect(0,0,600,200);
c.restore();
Everything we have done so far has been using images or shapes. It's been fairly high level. However, canvas also gives you direct access to the pixels if you want it. You can get the pixels for an entire canvas or just a portion, manipulate those pixels, then set them back. This lets you do all sorts of interesting effects.
Let's suppose we'd like to generate a checkerboard texture. This texture will be 300 x 200 pixels.
var data = c.createImageData(300,200);
for(var x=0; x<data.width; x++) {
for(var y=0; y<data.height; y++) {
var val = 0;
var horz = (Math.floor(x/4) % 2 == 0); //loop every 4 pixels
var vert = (Math.floor(y/4) % 2 == 0); // loop every 4 pixels
if( (horz && !vert) || (!horz && vert)) {
val = 255;
} else {
val = 0;
}
var index = (y*data.width+x)*4; //calculate index
data.data[index] = val; // red
data.data[index+1] = val; // green
data.data[index+2] = val; // blue
data.data[index+3] = 255; // force alpha to 100%
}
}
//set the data back
c.putImageData(data,0,0);
Pretty simple. We create a new buffer, loop over the pixels to set the color based on the x and y coordinates, then set the buffer on the canvas. Now you will notice that the even though we are doing two-dimensional graphics, the buffer is just a one dimensional array. We have to calculate the pixel coordinate indexes ourselves.
Now lets modify this to make it feel a bit more rough. Lets add a bit of noise by randomizing making some of the pixels a slightly different color.
if(val == 0) {
val = Math.random()*100;
} else {
val = 255-Math.random()*100;
}
There. That dirties it up a bit.
So that's generating new images with pixel buffers. We can also manipulate existing Canvas data. This means almost any sort of Photoshop filter or adjustment could be done with canvas. For example, suppose you want to invert an image. Inverting is a simple equation. A pixel is composed of RGBA component values, each from 0 to 255. To invert we just subtract each component from 255. Here's what that looks like:
var img = new Image();
img.onload = function() {
//draw the image to the canvas
c.drawImage(img,0,0);
//get the canvas data
var data = c.getImageData(0,0,canvas.width,canvas.height);
//invert each pixel
for(n=0; n<data.width*data.height; n++) {
var index = n*4;
data.data[index] = 255-data.data[index];
data.data[index+1] = 255-data.data[index];
data.data[index+2] = 255-data.data[index];
//don't touch the alpha
}
//set the data back
c.putImageData(data,0,0);
}
img.src = "baby_original.png";
Notice that we only modify the RGB components. We leave the Alpha alone since we only want to modify color. Here's what it looks like.
for(n=0; n<data.width*data.height; n++) {
var index = n*4;
var r = data.data[index];
var g = data.data[index+1];
var b = data.data[index+2];
var v = r*0.21+g*0.71+b*0.07; // weighted average
data.data[index] = v;
data.data[index+1] = v;
data.data[index+2] = v;
//don't touch the alpha
}
Notice that we don't choose a gray value by simply averaging the colors. I turns out our eyes are more sensitive to certain colors than others, so the equation takes that into account by weighting the green more than the other components. Here is the final result.
With pixel buffers you can pretty much draw or manipulate graphics any way you like, the only limitation is speed. Unfortunately manipulating binary data is not one of JavaScript's strong suits, but browsers keep getting faster and faster so some Photoshop style image manipulation is possible today. Later in the tools section I'll show you some libraries that make this sort of thing easier and faster.
Canvas doesn't define any new events. You can listen to the same mouse and touch events that you'd work with anywhere else. This is good and bad.
Remember that the canvas just looks like a rectangular area of pixels to the rest of the browser. The browser doesn't know about any shapes you've drawn. If you drag your mouse cursor over the canvas then the browser will send you standard drag events to the canvas as a whole, not to anything within the canvas. This means that if you want to do special things like making buttons or a drawing tool you will have to do the event processing yourself by converting the raw mouse events that the browser gives you to your own data model.
Calculating which shape is under the mouse cursor could be very difficult. Fortunately Canvas has an API to help: isPointInPath. This function will tell you if a given coordinate is inside of the current path. Here's a quick example:
c.beginPath();
c.arc(
100,100, 40, //40 pix radius circle at 100,100
0,Math.PI*2, //0 to 360 degrees for a full circle
);
c.closePath();
var a = c.isPointInPath(80,0); // returns true
var b = c.isPointInPath(200,100); // returns false
Now that we know how to draw lots of cool things, lets animate them. The first thing to know about animation is that it's just drawing the same thing over and over again. When you call a draw function it is immediately put up on the screen. If you want to animate something, just wait a few milleseconds and draw it again. Now of course you don't want to sit in a busy loop since that would block the browser. Instead you should draw something then ask the browser to call you back in a few milliseconds. The easiest way to do this is with the JavaScript setInterval function. It will call your drawing function every N msec. Lets try a simple example where we animate a rectangle across the screen.
var x = 0;
function drawIt() {
var canvas = document.getElementById('canvas');
var c = canvas.getContext('2d');
c.fillStyle = "red";
c.fillRect(x,100,200,100);
x+=5;
}
setInterval(drawIt, 100);Now you'll notice a problem. Our rectangle does go across the screen, updating by five pixels every 100 millesconds (or 10FPS), but the old rectangle is still there. It looks like the rectangle is just getting longer and longer. Remember that the canvas is just a pixel buffer. If you set some pixels they will stay there until you change them. So lets clear the canvas on each frame before we draw the rectangle.
var x = 0;
function drawIt() {
var canvas = document.getElementById('canvas');
var c = canvas.getContext('2d');
c.clearRect(0,0,canvas.width,canvas.height);
c.fillStyle = "red";
c.fillRect(x,100,200,100);
x+=5;
}
setInterval(drawIt, 100);So that's really all there is to animation. Drawing something over and over again. Lets try something a bit more complicated: a particle simulator. We want to have some particles fall down the screen like snow. To do that we will implement the classic particle simulator algorithm:
[ diagram: on each frame, create particle if needed, update particle, kill / recycle particle, draw particle]A particle simulator has a list of particles that it loops over. On every frame it updates the position of each particle based on some equation, then kills / creates particles as needed based on some condition. Then it draws the particles. Here's a simple snow example.
var canvas = document.getElementById('canvas');
var particles = [];
var tick = 0;
function loop() {
createParticles();
updateParticles();
killParticles();
drawParticles();
}
setInterval(loop,30);First we will create the essence of a particle simulator. It's a loop function that is called every 30 ms. The only data structure we need is an empty array of particles and a clock tick counter. Every time through the loop it will execute the four parts.
function createParticles() {
//check on every 10th tick check
if(tick % 10 == 0) {
//add particle if fewer than 100
if(particles.length < 100) {
particles.push({
x: Math.random()*canvas.width, //between 0 and canvas width
y: 0,
speed: 2+Math.random()*3, //between 2 and 5
radius: 5+Math.random()*5, //between 5 and 10
color: "white",
});
}
}
}The createParticles function will check if there are less than 100 particles. If so it will create a new particle. Notice that it only executes every 10th tick. This lets the screen start off empty and slowly build up, rather than creating all 100 particles right at the start. You would adjust this depending on the effect you are going for. I'm using Math.random() and some arithmetic to make sure the snow flakes are in different positions and don't look the same. This will make the snow feel more natural.
function updateParticles() {
for(var i in particles) {
var part = particles[i];
part.y += part.speed;
}
}updateParticles is super simple. It simply updates the y coordinate of each particle by adding it's speed. This will move the snow flake down the screen.
function killParticles() {
for(var i in particles) {
var part = particles[i];
if(part.y > canvas.height) {
part.y = 0;
}
}
}Here is killParticles. It checks if the particle is below the bottom of the canvas. In some simulators you would kill the particle and remove it from the list. Since this app will show continuous snow instead we will recycle the particle by setting the y back to 0
function drawParticles() {
var c = canvas.getContext('2d');
c.fillStyle = "black";
c.fillRect(0,0,canvas.width,canvas.height);
for(var i in particles) {
var part = particles[i];
c.beginPath();
c.arc(part.x,part.y, part.radius, 0, Math.PI*2);
c.closePath();
c.fillStyle = part.color;
c.fill();
}
}Finally we draw the particles. Again it's very simple: clear the background then draw a circle with the current particle's x, y, radius, and color.
Now here's what it looks like
What I love about particle simulators is that you can create very complicated and organic, natural looking animation with very simple math, combined with a bit of carefully chosen randomness.
The final major kind of animation is sprite animation. So what is a sprite?
A sprite is a small image that you can draw quickly to the screen. Usually a sprite is actually cut out of a larger image called a sprite sheet or master image. This sheet might contain multiple sprites of different things, like the different characters in a game. A sprite sheet might also contain the same character in different poses. This is what gives you different frames of animation. This is the classic flip-book style of animation: simply flip through different drawings over and over.
Sprites are good for a few of things.
Sprites are easy to draw using the drawImage function. This function can draw and stretch a portion of an image by specifying different source and destination coordinates. For example, suppose we have this sprite sheet and we just want to draw the sprite in the center (5th from the left).
We can draw just this sprite by specifying source coordinates:
context.drawImage(
img, // the image of the sprite sheet
65,0,13,13, // source coordinates (x,y,w,h)
0,0,13,13, // destination coordinates (x,y,w,h)
);
As you can see in the full sprite sheet, this is really the same object drawn in different frames of an animation, so now let's flip through the different sprites to make it be animated. We'll do this by keeping track of the current frame using a tick counter.
var frame = tick % 10;
var x = frame * 13;
context.drawImage(
img, // the image of the sprite sheet
x,0,13,13, // source coordinates (x,y,w,h)
0,0,13,13, // destination coordinates (x,y,w,h)
);
tick++;
Every time the screen is updated we calculate the current frame animation by looking at the tick. Doing a mod (%) 10 operation means the frame will loop from 0 to 9 over and over. Then we calculate an x coordinate based on the frame number. Then draw the image and update the tick counter. Of course this might go too fast, so you could divide the tick by 2 or 3 before the mod to make it run slower.
Now lets go back to some hands on work. For this section we are going to build a simple game. This will show you how to use animation, keyboard events, and use a simple particle simulator for explosions.
Hands On Lab #2: Space N Vaders
Canvas also supports composite modes. These are similar to some of the blend modes you will find in Photoshop. Every time you draw a shape each pixel will be compared to the existing pixel, then it will calculate the final pixel based on some equation. Normally we are using SrcOver, meaning the source pixel (the one you are drawing) will be drawn over the destination pixel. If your source pixel is partly transparent then the two will be mixed in proportion to the transparency. SrcOver is just one of many blend modes, however. Here's an example of using the lighter mode when drawing overlapping circles. lighter will add the two pixels together, with a maxium value of white.
c.globalCompositeOperation = "lighter"; //set the blend mode
c.fillStyle = "#ff6699"; //fill with a pink
//randomly draw 50 circles
for(var i=0; i<50; i++) {
c.beginPath();
c.arc(
Math.random()*400, // random x
Math.random()*400, // random y
40, // radius
0,Math.PI*2); // full circle
c.closePath();
c.fill();
}
Canvas also supports shadows, similiar to CSS. You can set the color, offset and blur radius of the shadow to simulate different effects. This is an example of doing a white glow behind some green text.
c.fillStyle = "black";
c.fillRect(0,0,canvas.width,canvas.height);
c.shadowColor = "white";
c.shadowOffsetX = 0;
c.shadowOffsetY = 0;
c.shadowBlur = 30;
c.font = 'bold 80pt Arial';
c.fillStyle = "#55cc55";
c.fillText("ALIEN",30,200);
Now that you know a lot about how canvas works, lets explore what it's actually good for and some useful libraries.
RGraph is a free for personal use charting library for canvas. It has many different chart forms.
www.rgraph.net
ZingChart is a hosted charting library with a visual builder. It renders in many different output formats, including Canvas, and can handle large datasets.
http://www.zingchart.com/
Wolfenstein 3D recreated in canvas.
Opera Dev Article
Akihabara Game Engine
www.kesiev.com/akihabara
ImpactJS: fast commercial game engine
impactjs.com
Cocos2D: partial javascript port of the Cocos iPhone SDK
cocos2d-javascript.org
Pirates Love Daises is a tower defense game done entirely in canvas.
PiratesLoveDaises.com
Muro: Deviant Art's webbased painting program.
deviantart.com
SketchPad: another drawing program with a very classy UI
mugtug.com/sketchpad/
Ben Joffe's canvas font script. Converts a font on your computer into
an image which can be rendered with canvas. This lets you use a custom font on computers that don't have that actual font installed.
benjoffe.com
A canvas enriched children's poem. The text is markup and the graphics are in a transparent canvas.
Josh On Design
EaselJS: A graphics library loosely based on Flash's display list.
Easel JS
A javascript port of the Java Processing graphics library. Great for
interactive displays and art.
Processing JS
Kapi: a keyframing javascript library.
JeremycKahn.github.com/kapi/
canvg: an SVG renderer built with canvas
code.google.com/p/canvg/
Pixastic is a photo editor and image processing library. It has tons of Photoshop style filter effects
Pixastic.com
Hype by Tumultco, a commercial drawing and animation tool
which outputs straight HTML 5
tumultco.com/hype/
Amino : open source JavaScript and Java scenegraph.
Go Amino.org
Leonardo Sketch: open source drawing tool which outputs to canvas and Amino code, among other formats. It is extensible and has some neat social features.
LeonardoSketch.org
Leonardo Sketch & Amino
show an example of drawing something and exporting to canvas code
show an example of drawing something and exporting to amino code
show amino code, explain it's a scenegraph for canvas
with events, smart repainting, and animation interpolation.
open source goamino.org
Now, let's talk about mobile devices and optimization. There is no mobile version of canvas, there's just canvas. It's the same API on desktop and mobile devices. Mobile devices are sometimes missing features, however, and are usually slower; but the same could be true on older desktops and browsers. So whenever you are making a canvas app it's important to consider performance and different ways to optimize your code.
The general mantra for performance is draw less.
don't draw hidden things. If you have four screens of information but only one is visible at a time, then don't draw the others.
use images instead of shapes. If you have some graphic that won't ever change or be scaled, then consider drawing it into an image at compile time using something like photoshop. In general images can be drawn much faster to the screen than vector artwork. This is especially true if you have some graphic that will be repainted over and over again like a sprite in a game.
cache using offscreen canvases. You can create new instance of the canvas object at runtime that aren't visible on screen. You can use these offscreen canvasas as a cache. When your app starts draw graphics into the offscreen canvas then just copy that over and over again to draw it. This gives you the same speed as using images over shapes, but you are generating these images at runtime and could potentially change them if needed.
image stretching. Since we are using images for lots of things already, consider stretching them for effects. Most canvas implementations have highly optimized code for scaling and cropping images so it should be quite fast. There are also several versions of drawImage that let you draw subsections of an image. With these apis you can do clever things like caching a bunch of sprites into a single image, or wildly stretching images for funky effects. [screenshots]
only redraw the part of the screen you need. Depending on your app it may be possible to just redraw part of the screen. For example, if I have a ball bouncing around I don't need to erase and redraw the entire background. Instead I just need to redraw where the ball is and where it was on the previous frame. For some apps this could be a huge speedup.
Draw fewer frames Now that you are drawing as little per frame as possible try to draw fewer frames. To get smooth animation you might want to draw 100fps, but most computers max out at a 60fps screen refresh rate. There's no point in drawing more frames because the user will never see them. So how do you sync up with the screen refresh? Mozilla and WebKit have experimental apis to request that the browser call your code on the next screen refresh. This will replace your call to setInterval or setTimeout. Now the browser is in charge of giving you a consistent framerate, and it will ensure you don't go over 60fps. It can also do smart things like lowering the framerate if the user switches to a different tab. Mobile browsers are starting to implement this as well so background apps will be throttled back, saving on battery life.
The best way to draw less is to not draw it at all. If you have a static background then move it out of canvas and draw it with just an image in the browser. You can make the background of a canvas transparent so that a background image will show through. If you have large images to move around you may find they move faster and smoother by using CSS transitions rather than doing it with javascript in the canvas. In general CSS transitions will be faster because they are implemented in C rather than JS, but your mileage may vary, so test test test. Speaking of which: Chrome and Mozilla have great tools to help you debug and test your JavaScript. [names? examples?]
pixel aligned images. One final tip, in some implementations images and shapes will draw faster if they are draw on pixel boundaries. Some tests show a 2 to 3x speedup [verify] on the ipad canvas impl if you pixel align your sprites.
http://jsperf.com/drawimage-whole-pixels http://sebleedelisle.com/2011/02/html5-canvas-sprite-optimisation/
An MP3 style visualizer using buffer to buffer copying
Visualizer Test
An experimental Mozilla api for directly generating and manipulating audio samples.
View Live at Nihilogic.dk
Mozilla Audio Data API
Metaballs demo using WebGL
ro.me
3D Solar System using WebGL
spacegoo.com
Blow video up into fragments while playing it!
craftymind.com
[if we still have time we will do a 3rd handout using leonardo and amino to draw stuff and create a simple game draw shapes and text. make text be rasterized w/ custom font. cache some gfx in buffer nodes. cache some gfx as images. ]