So many cool and useful technologies are unveiled at SIGGRAPH every year, this year at SIGGRAPH 2009 was no different.  Khronos Group, behind the new guidance of OpenGL, OpenGL ES, OpenCL, OpenVG, COLLADA etc, came another big announcement about hardware rendering within the browser.  WebGL is now an official standard being developed at Khronos Group to bring javascript control of OpenGL to browsers… Wow!

Ok so this was officially announced at the GDC in March but limited information, but now it has been slated for an official public standard in early 2010. Shortly after the announcement at the GDC we saw Google o3D appear doing exactly that, controlling OpenGL through Javascript in the browser but it was still largely software/harward hybrid rendered. Google, Mozilla, Opera are part of the companies supporting WebGL which is great for browser support, also NVIDIA, AMD and Ericsson are in on it.

Khronos Details WebGL Initiative to Bring Hardware-Accelerated 3D Graphics to the Internet

JavaScript Binding to OpenGL ES 2.0 for Rich 3D Web Graphics without Browser Plugins;
Wide industry Support from Major Browser Vendors including Google, Mozilla and Opera; Specification will be Available Royalty-free to all Developers

4th August, 2009 – New Orleans, SIGGRAPH 2009 – The Khronos™ Group, today announced more details on its new WebGL™ working group for enabling hardware-accelerated 3D graphics in Web pages without the need for browser plug-ins.  First announced at the Game Developers Conference in March of 2009, the WebGL working group includes many industry leaders such as AMD, Ericsson, Google, Mozilla, NVIDIA and Opera.  The WebGL working group is defining a JavaScript binding to OpenGL® ES 2.0 to enable rich 3D graphics within a browser on any platform supporting the OpenGL or OpenGL ES graphics standards.  The working group is developing the specification to provide content portability across diverse browsers and platforms, including the capability of portable, secure shader programs.  WebGL will be a royalty-free standard developed under the proven Khronos development process, with the target of a first public release in first half of 2010. Khronos warmly welcomes any interested company to become a member and participate in the development of the WebGL specification.

Google released O3D this year and there are great strides in 3d within the browser from game engine wrapper technologies such as instant action technology, gaim theory engine (now owned by id Software and runs Quake  Live, hardware rendered Unity 3D (and Torque 3D coming soon), and Flash software rendered  3d engines Papervision 3D, Away 3D, Sandy (Sandy also released a haXe version that exports a javascript version) and others.  But it looks like the movement is to bring OpenGL to the web as a standard under the name WebGL, this would be great!  There would still be lots of times where plugins are better now and in the near future but the path is a good one. Having a software/hardware rendering hybrid like Google O3D for broad video card support (some of the painful older intel cards), or using a plugin like Unity3D, Torque 3D or wrapper technology for bigger engines is a good idea for the time being. But the future is grand in this area.

I think that Google O3D and OpenGL ES success on iPhone games probably combined to get this in motion.  OpenGL and very basic video cards are now standard in most machines out there.  Unity3D actually published hardware statistics on casual gamers (web-based games) ever so kindly and shows that even though there are some older Intel cards out there, for the most part machines nowadays have a video card capable of supporting at least low-poly 3d and hardware supported 2d rendering in real-time for games, user interfaces and more.

This is exciting news, it appears the movement of the web gaming market is getting much more capable and is accelerating the innovation of hardware accelerating the web.

This is a very cool project called V8-GL.  It is an OpenGL engine with 80% of the API converted to run on the V8 Javascript engine, the same engine that runs Google Chrome.

This is exciting as more productive languages like Javascript get speed boosts from engines like V8 and are capable of manipulating more complex systems like OpenGL.  Google is also pursing this in the browser with O3D with javascript manipulation of hardware rendering.  Also, a Google funded project called Unladen Swallow is converting Python to the LLVM virtual machine, so that it can have increasing speeds to compete with gcc speeds.

Making things easier to produce and control with more simplified and minimal languages like Javascript, Python and Actionscript etc that control more complex systems, that typically you would need to invest more time in such as a platform on C++ is the goal. V8-GL has this goal in mind.

V8-GL from the author states:

V8-GL intends to provide a high-level JavaScript API for creating 2D/3D hardware accelerated desktop graphics.

In other words, you can hack some JavaScript code that opens a desktop window and renders some 3D hardware accelerated graphics. Bindings are made using the V8 JavaScript engine.

• V8-GL blog post
• V8-GL source on Github
• Also check out the SpiderMonkey jslibs project with other javascript wrappers for libraries like ode, sdl and more using the engine by Mozilla.

Google has a few things going for 3d in the browser, not just 3d but hardware rendering in the browser.  They previously had native client which allows you to run code via a plugin proxy with a sample running Quake.  They also had Lively which was a virtual world plugin that was shut down a few month after it started.

Now they are also making and releasing an O3D plugin that looks to be another way to do web 3d scenes and games although it is a very early stage. They appear to want to have an open discussion about how best to add hardware rendering to the web.  Their approach uses a javascript api to control the browser plugin and the O3D control is essentially just a renderer.

This won’t change anything now as Unity3D, Flash 3D pseudo engines, even Director 3D still are the top choices for games, apps, and interactives that need effects and possibly hardware rendering. But it is interesting that Google is essentially re-entering this debate after ditching on Lively and they must see some benefit to having a discussion about 3d on the web and 3d standards in general.  I know they have lots of models and tools with SketchUp and Google 3D warehouse so who knows maybe they will take it over by being standards, open and information based.

What is O3D?

O3D is an open-source web API for creating rich, interactive 3D applications in the browser. This API is shared at an early stage as part of a conversation with the broader developer community about establishing an open web standard for 3D graphics.

Get involved

• Download the plug-in (Windows and Mac) and explore the samples to see O3D’s capabilities. Linux users, see these instructions.
• Read the Technical Overview and Developer’s Guide to learn how to get started.
• Join the O3D developer groups to provide feedback.

One thing is for sure, 3d development is still old school proprietary lock in in most cases.  Working with 3d and tools like Maya, 3dsmax and others they have always been very non standard.  From file formats to interfaces to even basic movements, all different.  The general maths of 3d are the same and so should 3d pipelines.  Formats like COLLADA are nice because they are starting to open up 3d pipelines and content creation but COLLADA still has many porting issues.  FBX file format is another that is really useful and common making pipelines in Unity 3D, for instance, very nice. But it is owned and run by Autodesk who owns all the 3d apps (Maya, 3dsmax, SGI) and I am a bit leary of that method.  But in the end 3d pipelines and rendering will be somewhat standardized and maybe the web will be hardware rendered one day.  In most cases it is not needed, but for gaming, immersion, demos and other entertainment it could benefit heavily from a more standardized 3d pipeline and methods.

Torque3D seems to have full featured browser surfaces that you can use in the 3d engine seamlessly, see the video below (at :38). This is amazing stuff.

Making games that integrate content from the web is especially required these days.  It is a difficult thing to do within the 3d render because of all the plugins, styling etc that needs to be rendered on a 3d surface.  Well Torque3D has a killer feature in that it supports entirely full features browser render on a 3d surface.  So now you can integrate html content, flash video etc in your game easily.

You can play content in flash player content easily and have stripped down html but it is limited, you can play videos and have content in Unity3d but it is limited, even larger engines like Unreal 3 have difficulty handling flash and html content.  If this is a good implementation Torque3D has a killer feature on their hands! Flash is commonly used as user interface elements and content within games but it can be challenging.  This is pretty exciting if it works as advertised. Think of how cool all the little consoles, mini-games and controls in 3d games could be in flash easily.

See at :38 in the video…

GDC 2009 – Torque 3D Web Demo from GarageGames on Vimeo.

Away3D was updated to Flash 10 earlier this month.  Flooded with final semester, massive workload and all the conferences #swsx, #gdc, #mix09 etc I missed the announcement.  

It is looking pretty sweet with this mustang demo showing off the update.  Pixel bender has provided a performance update for effects that shows nicely here.

The above demo shows some of what is possible with the update: normalmapping with ambient, diffuse and correctly normalised specular shading, without the need for layers. As if that’s not enough, a further Pixel Bender shader is applied to the view to create a HDR (High Dynamic Range) effect on the highlights, something usually seen in much more 3d-rich console games. Special thanx go to Eddie Carbinfor donating an excellent normalmapped mustang model, and David Lenaerts for writing the HDR filter.

Airtight Interactive did a comparison a while back where FP10 rendering was faster against the olde version of Away3D, this update may change that will have to take a look.  All away3d needs is a designer

It appears Torque3D is going to compete with Unity3D in a browser based 3d plugin front with a Torque3D toolset and pipeline that target the web.

They announced this on gamasutra recently on how instantaction and torque3d technology is similar yet different. InstantAction is an engine wrapper where the web plugin for Torque3D is more tighly coupled with the Torque3D engine. Instantaction.com technology allows you to wrap an existing game engine for the web which is a competitor with the gaimtheory engine that is used on quakelive.com.

I have been a garage games torque developer and member since 2003 and worked/bought with each engine they have put out from the old school Torque Engine, to Torque Engine Advanced for various game development projects and now they are throwing in on what appears to be based on their instantaction.com technology but using the Torque3D engine. This is very interesting, they might even have a channel/appstore to release games on instantaction.com?

Unity3D has recently taken my time in the full immersion 3d for the web space, the mono engine that runs the scripting is a huge feature. Coding in C#, Boo and javascript is great, and the pipeline for Unity3D is unmatched.

A bit of history, I have been interested in this since Director introduced 3d in director 8.5 in 2001 (how was that not a major revision I don’t know) and the killer Havok 3d physics engine within it. Way ahead of its time. But Director 3d was extremely limited with w3d (not even a decent blender exporter) and it literally has not advanced since that time in terms of ability to develop better for it and the IDE. It was trapped in this little IDE and quirky Lingo language. They tried to save it with javascript, a valiant effort but it still withered due to lack of openness of development for the player (a mistake they aren’t making with Flash now at Adobe). So making full immersion 3d games was not really ready for the web, Director was notorious at crashing browsers and took way too many broken plugins to get a basic engine.

So I went to mods in HL (quake 2 engine), Unreal and when I realized there was no way me or my friends could foot the license fees of either engine we went to Torque in 2003. It was great, large terrains and highly compact engine because it had to run Tribes with 64 players years before anyone else approached 64 players and arguably still played better than 64 player fps now. It was affordable. I have been interested in the movements to make torque an active x control in 2005/6 and Think Tanks did just that. It was very nice, I thought soon after it would be everywhere. But it has taken until now and a new engine called Torque3D before this has been realized years later. This is hard stuff to get right creating a plugin that works cross browser and performs well (browsers had to catch up as well). The culmination of that technology progression has happened and 3d on the web for game development appears to be bigtime in 2009.

Unity3D meanwhile since 2005-6 has gotten it nearly all right so far for a few years now, especially the pipeline and the webplayer. Torque has always had an poor pipeline, not as bad as writing your own engine from scratch and all the tools but in the early days pretty close. Proprietary formats like dts for models made finding the right exporter tasking. There were just so many walls in what was supposed to be a pipeline, largely due to support for formats that were small enough for slower networks and machines of the past. Unity3D gets all this right from the start, pipeline is not an issue. Torque3D seems to address this with their new tools, support for Collada, but unfortunately still scripted with TorqueScript. If Torque3D could wire in Mono and the capability to code in Javascript, C#, Boo or other Mono languages this would have been cool, or at least a semi-standard scripting language like Javascript, Python etc that would be great. TorqueScript was a big feature and UnrealScript is very similar in how it interacts with the engine, but these days we want standard languages that have engine features built in. Maybe this will happen down the road, but the format support is very nice.

For 3d, prior to Unity3D, the web was still owned by Director but that has changed in the last few years. It won me over for immersive 3d games that are web capable and able to port to other platforms and markets. Casual is still done with Flash and mobile space is targeting iphone. Unity3D can run on web, iPhone, Wii, desktop. Torque3D can run on web and also has paths to desktop, iPhone, Wii, XBOX360. You see what is happening here? It is very cool indeed. No doubt the competition in this area is getting to a point where some good innovation and happenings are taking place, what are you going to do with it?

Here is a video with more information on Silverlight 3 Beta features that are matches of the latest Flash killer features in pixel shaders, 3d planes (ability to create pseudo-3d engines like papervision3d), local saving, pixel operations/bitmap handling, local messaging (silverlight to silverlight – like localconnection), out of browser desktop running ability of SL3, SEO and search indexing capabilities / deep linking navigation and more.

Video of the features of SL3 Beta, Demos, at a Slow Pace from #mix09

A few points after the video and taking a tour of the features.

The pixel shaders are written in HLSL (shader 2), however they are compiled to byte code and do not currently use the gpu for rendering. While the pixel shaders are very cool and the language to write them is standard pretty much for shaders in HLSL shader model 2 DirectX-based (the other is GLSL OpenGL based) they have not allowed this byte code to run on the GPU… yet. Here Flash and Pixel Bender actually are ahead there.

Although there are 3d planes which is very exciting, no good pseudo 3d engine exists yet matching the 3 in flash (papervision3d, away3d, sandy). When SL3 comes out I am sure we will see a few emerge or build them ourselves because this iteration of SL3 looks pretty fun.

Pixel based operations will be a huge advancement much like it was in earlier flash versions as it adds some demo scene type abilities and experiments with pixels that are fun.  This also lends to doing cool things like shaders, effects, AR, face recognition, motion detection etc.

Effects like Blur and Drop shadow are good and the ability to add custom ones, great. However currently they are pretty performance intensive.  They are also in Flash but there needs to be some refinement in SL3 effects before launch.

Desktop runnable apps in out of browser will be nice and this is a direct compete with Adobe AIR which was a surprise.

Local Communication supports desktop to browser communication.

Isolated storage (similar to shared objects) supports 1MB in browser, 25MB out of browser defaults.

This version of silverlight is really a 1.0 version as typical with most software.  Version 2 or version 3 is usually what the initial design goals pan out.  Much like the latest unity3d version (2.5) that has windows support and the iPhone SDK 3.0 that both came out this week, even actionscript 3 compared to earlier versions, these toolkits are finally iterated enough that they are really solid platforms for building cool stuff on and become platforms.  The next version of all these could be very, very dangerous.

Huge missing features:

Although there are some great features in SL3 beta, it is still not done and it is still missing some key components that Flash has which make it very attractive in the interactive space.

• Camera and Microphone support - Macromedia hired one of the smartest dudes around in Jeremy Allaire back in flash 6 days to help add support for Flash Communication Server (Flash Media Server now) Camer and Microphone support.  One of the best R&D periods at Macromedia. SL needs this soon.
• Printing support – what was long a problem in Flash is so in SL, there is no good printing support
• No GPU usage for Pixel Effects/Shaders – (neither flash nor silverlight support hardware accelerated shaders in PixelEffects/Pixelbender – Pixel Effects/Shaders need GPU support (see Kevin Goldsmith’s article on GPU mixed with CPU and how this may or may not be good. However processors are speeding up and multi-core helps software rendering, the quality of GPU is well beyond what software rendering can deliver for a few years to come at least while architecture advances, probably more like 5-10 years.
• No UDP plans yet - Adobe has RTMFP, SL sockets has no public plans for adding UDP that I have seen
• No Alpha Channel in Video - You can do this with a shader though but not supported by default.

Silverlight 3 Video

Flash has the upperhand in video and probably will still even though SL3 has H.264.  Flash added this at the same time and though they still have FLV which revolutionized web video they are now much broader in support in video than SL3.  Silverlight has H.264 and VC1 support (their own FLV like codec).  Still pretty cool a couple years ago there was no HD on the web now everyone has it in H.264 video support.

Currently nothing innovative, mainly catchup still, but here are some options

Silverlight 3 beta and the video below the features and highlights will look very similar to flash and flash community advancements over the last couple years.  There is no innovation just yet.

But where that could happen is in socket support with UDP. Flash has moved on this in RTMFP and the beginning of larger scale networking support with UDP with samples like stratus.  This is a huge differentiating feature for what I think will be game changer on the web (it already is on desktop mmos) in real-time or closer to real-time support for larger sets of users in online games like MMOs or virtual communities, even tools to make request based real-time sites like micro-blogging faster and able to handle more users (right now it is very linear if users get many followers, UDP will allow a better distributed framework for messaging).

Local Storage

Silverlight and Unity3D all need this, Flash could use better support for this.  Local saving of a files for cache beyond the internet cache and greater than the 1MB/25MB limits of SL3 IsolatedStorage.  This is an issue when you are making large scale games in that you need to save lots of assets to a client but to make it economical you want ot save more than the default internet cache amount. Flash Shared Object (Local) allow you to do this somewhat but it would be great to have a way to just download files for cache (upon user agreement) to store assets in bulk of allowable types (images, video, models, bundles) to the file system.

Hardware rendering for 3d support and UDP support will put Flash and SL3 on par with the killer Unity3D kit for making online web games and other activex/plugins like instantaction that allow you to do these things already.

The one thing SL has over Flash

Flash and Flex are great. But there is this massive division in the community and marketing of Flash. Silverlight is entirely unified and this has much to do with starting clean at a time that interactive development is heading more into a technology and developers control. Flash and Flex need to bring it together. AS3 has been out long enough that the people with skills have hopped on and taken it to a new level, mainly from programmers. If Adobe created a version of Flash that was a new IDE and consolidated Flex and Flash into just Flash, made the IDE as powerful as FDT or FlashDevelop3 there could be hope to bring the platform together. I understand they had to work it in slowly because it was a designers platform really (even though coders still pushed the limits in games and apps built on it) so they had to tip toe carefully on this to not alienate people. But now I think the division is a serious problem with the platform and must be addressed, noone expected Silverlight to be this quick on at least SL3 features. And even though the initial approach might have been bad as SL1 was a huge letdown, Microsoft does not give up and you can see in the XBOX360 and DirectX that they are very pursuant. DirectX really didn’t become huge until version 7 so these guys won’t relent.

I am not a huge fan of using the proprietary tools. Even in Flash I use as much open source as I can even though the player is locked, but Moonlight is something that trails Silverlight development and is a very unique thing in both open source and cross platform/multiplatform development. It is a clear relationship and aims to make Silverlight run on multiplatform mono including Linux. This could win out in the end who knows.

Futures

Great iterations of software happened this week in the latest unity3d version (2.5) that has windows support and the iPhone SDK 3.0 and now SL3 is quite a surprise in feature set.

I have been really busy this week just delving into all them and hope to start making more cool and useful projects in them.  The best part is right now is great to be an interactive or game developer as all major software companies and markets are focused on retaining good developers.  I don’t’ recall a time other than the beginning the the web virtual land rush that has so many options and markets that skilled developers and designers can choose from.  Good times.

This is fantastic news!  Unity3D 2.5 has been released a bit early before GDC and it includes the game changing windows build environment and IDE.  This now opens up the Unity 3D market by a huge factor and Unity3d will find its way into many gaming companies that are heavily invested in Windows. We should see the amount of unity 3d players and content ramp up quite a bit this year.

• Forum Announcement
• What’s new in Unity3d 2.5
• Download the 2.5 update
  
  • Mac OS X: http://unity3d.com/unity/unity.dmg
  • Windows: http://unity3d.com/unity/UnitySetup.exe <- well look at that

Dont’ get me wrong I love my Mac Book Pro and continue to use it for development in Unity3D, iphone sdk etc.  But being able to use my windows machine as a dev box is great and I know this will be huge for many windows users not wanting to shell out the $1500 for the unity license AND $3000 for a decent macbook.  The cost wall has been lowered and it is a great investment if you are an indie game developer or a large game developer.

I usually only get excited about open source tools on this blog because they help everyone with skills have access, I hope one day there will be an open source 3d browser based engine as well.  Right now though there is nothing price competitive other than maybe torque instantaction plugin or the gaim engine (quakelive) that comes close with hardware rendering and none of them beat Unity in ease of pipeline.

One thing about Unity3d is it was a game engine from day one, it is only a game engine.  It isn’t like Director or Flash that are also application development kits, rich internet application kits etc.  This is pure game engine baby with hardware rendering for the win. Also, if you want to make 3d games for the iPhone without going hardcore OpenGL ES you can do so with an additional license that opens up your distribution channel to the iphone, flash can’t do that currently unfortunately.

And so it begins… GAME ON!

Libspark from Japan is a treasure trove of great flash advancements, they seem to realize the great things that can come from porting in existing solid libraries from C/C++ etc into flash and have been scoring lately including augmented reality in flash porting the ARToolkit to FLARToolkit.  Recently a port of openCV for as3 called Marilena was found and it is for object detection and decent facial recognition (it is a computer vision library from intel) considering the processing power needed to do this.

Face Detection: Here is the sample included with Marilena showing facial detection on an image.

Lots of recent action has blown up on this front from Mr doob, quasimondo (optimizing the Marilena classes for better performance) and Boffwswana. Also there is a kit called deface by sshipman that is the first foray into this a year ago doing similar things but it was just a bit before it’s time and a bit slow in previous versions of flash, it performs decent now in this sample. Flash 10 performance of the AVM2 and future directions with Alchemy will lead to more interesting stuff just like this.

Mr. doob head tracking sample, be sure to check lots of other examples there

Boffswana example of head tracking Johnny Lee Wii style with only a webcam and flash, no wiimote needed since it uses facial detection to check where you are and how close you are in the screen and then moves accordingly.

This is stemming from the recent explosion of the FLARToolkit and augmented reality in flash as well as the gimmicks used by Nintendo with the wii and Johnny Lee’s great head tracking advancements. Porting great libraries to flash seems to be the phase we are entering now judging by the recent excitement around Adobe Alchemy and the LLVM along with the lead from the libspark.org contributors. We have also seen this heavily last year in ports of Box2D for 2d physics and other toolkits using established working code and porting that to flash now that is is mostly capable of handling the performance.

OpenCV (Open Computer Vision Library by Intel) is quite a powerful platform that allows you to do all this and now it is available in flash. There are other great libraries for nearly all platforms now. I have done some previous work with Aforge which is also a port of OpenCV mainly for motion detection. This was always around but not until the recent performance updates and the innovation that has come with Alchemy and the thinking that goes along with that (porting in libraries to flash from C/C++ etc), has allowed this to flourish in flash and thus the web.

The amazing new things we can do with flash by porting in existing libraries is only going to get more intense as alchemy and flash 10 are even more mainstream.  It is almost as if Flash will eventually just become a web renderer and simplified front end to many great toolkits that exist in more native environments like C/C++ but with the speed and distribution access of the web with Flash.  Exciting times ahead.

Looks like it is a javascript day here at *drawlogic.  Here is an interesting example with some demos of a javascript and canvas based pseudo 3d engine. Anything this cool you know it has to be from Japan.
Also of note, it has been rumored that Silverlight 3 will have fully hardware accelerated 3d and canvas and javascript engines are getting much faster with great demos like this.  Adobe needs to leap into hardware acceleration for flash on a broader scale soon.

But I digress, this demo it appears, was inspired by Papervision3D due to the naming and the javascript reference of “parpevision.js“.  I wasn’t able to find much more information about this but it is very well done and this example even shows some environment mapping. It is not close to flash pseudo-3d engines like Papervision3D yet but at the rate of javascript engine development lately this could rival flash AVM2 in the next couple of years.

Demos

• Cloth
• Miku Character
• Enviroment Mapping

Code

• Sample iPod iTouch Mesh data (meshdata.js) – contains models
• Code from the main engine (parpevision.js)
• Sample Viewport and main app (touch.js from demo 3)

Here is the code for the parpevision.js file and the mini engine, it is an MIT license. (more…)