So many cool and useful technologies are unveiled at SIGGRAPH every year, this year at SIGGRAPH 2009 was no different. Khronos Group, behind the new guidance of OpenGL, OpenGL ES, OpenCL, OpenVG, COLLADA etc, came another big announcement about hardware rendering within the browser. WebGL is now an official standard being developed at Khronos Group to bring javascript control of OpenGL to browsers… Wow!
Ok so this was officially announced at the GDC in March but limited information, but now it has been slated for an official public standard in early 2010. Shortly after the announcement at the GDC we saw Google o3D appear doing exactly that, controlling OpenGL through Javascript in the browser but it was still largely software/harward hybrid rendered. Google, Mozilla, Opera are part of the companies supporting WebGL which is great for browser support, also NVIDIA, AMD and Ericsson are in on it.
Khronos Details WebGL Initiative to Bring Hardware-Accelerated 3D Graphics to the Internet
JavaScript Binding to OpenGL ES 2.0 for Rich 3D Web Graphics without Browser Plugins; Wide industry Support from Major Browser Vendors including Google, Mozilla and Opera; Specification will be Available Royalty-free to all Developers
4th August, 2009 – New Orleans, SIGGRAPH 2009 – The Khronos™ Group, today announced more details on its new WebGL™ working group for enabling hardware-accelerated 3D graphics in Web pages without the need for browser plug-ins. First announced at the Game Developers Conference in March of 2009, the WebGL working group includes many industry leaders such as AMD, Ericsson, Google, Mozilla, NVIDIA and Opera. The WebGL working group is defining a JavaScript binding to OpenGL® ES 2.0 to enable rich 3D graphics within a browser on any platform supporting the OpenGL or OpenGL ES graphics standards. The working group is developing the specification to provide content portability across diverse browsers and platforms, including the capability of portable, secure shader programs. WebGL will be a royalty-free standard developed under the proven Khronos development process, with the target of a first public release in first half of 2010. Khronos warmly welcomes any interested company to become a member and participate in the development of the WebGL specification.
Google released O3D this year and there are great strides in 3d within the browser from game engine wrapper technologies such as instant action technology, gaim theory engine (now owned by id Software and runs Quake Live, hardware rendered Unity 3D (and Torque 3D coming soon), and Flash software rendered 3d engines Papervision 3D, Away 3D, Sandy (Sandy also released a haXe version that exports a javascript version) and others. But it looks like the movement is to bring OpenGL to the web as a standard under the name WebGL, this would be great! There would still be lots of times where plugins are better now and in the near future but the path is a good one. Having a software/hardware rendering hybrid like Google O3D for broad video card support (some of the painful older intel cards), or using a plugin like Unity3D, Torque 3D or wrapper technology for bigger engines is a good idea for the time being. But the future is grand in this area.
I think that Google O3D and OpenGL ES success on iPhone games probably combined to get this in motion. OpenGL and very basic video cards are now standard in most machines out there. Unity3D actually published hardware statistics on casual gamers (web-based games) ever so kindly and shows that even though there are some older Intel cards out there, for the most part machines nowadays have a video card capable of supporting at least low-poly 3d and hardware supported 2d rendering in real-time for games, user interfaces and more.
This is exciting news, it appears the movement of the web gaming market is getting much more capable and is accelerating the innovation of hardware accelerating the web.
Haxe Sandy is a version of Sandy that can export to an experimental Javascript 3D engine taking advantage of the <canvas> element. There are some great demos that run smoothly in canvas capable browsers and very smooth in Chrome.
Sandy was actually the first open source 3d engine in flash, maybe this will be a trend building in haXe for export to flash and javascript? It certainly looks like a great start and would make a very nice platform for 3d on the web allowing Sandy or other flash libraries to run in Flash and Javascript by writing in an abstraction platform like haXe. Other libraries like Motor2, Physaxe, haxe3D, PureMVC and more have haXe versions. Still very experimental but a possible need when Flash and canvas are both in the market in the future. Right now it is still all Flash.
A very nice feature of this library, in addition to being coded cleanly and as3 style, is the ability to have the Twitter OAuth page render inside of Flash.
Instead of opening the Twitter authorization page in the browser the library also contains OAuthLoader which is a wrapper around HTMLLoader which enables to directly show the authorization page within an AIR window:
// use this in the requestTokenHandler instead of navigateToURL
var loader:OAuthLoader = new OAuthLoader();
loader.load(request);
loader.percentWidth = 100;
loader.percentHeight = 100;
var w:Window = new Window();
w.width = 800;
w.height = 400;
w.title = req.url;
w.addChild(loader);
w.open();
Making things easier to produce and control with more simplified and minimal languages like Javascript, Python and Actionscript etc that control more complex systems, that typically you would need to invest more time in such as a platform on C++ is the goal. V8-GL has this goal in mind.
V8-GL from the author states:
V8-GL intends to provide a high-level JavaScript API for creating 2D/3D hardware accelerated desktop graphics.
In other words, you can hack some JavaScript code that opens a desktop window and renders some 3D hardware accelerated graphics. Bindings are made using the V8 JavaScript engine.
Google has a few things going for 3d in the browser, not just 3d but hardware rendering in the browser. They previously had native client which allows you to run code via a plugin proxy with a sample running Quake. They also had Lively which was a virtual world plugin that was shut down a few month after it started.
This won’t change anything now as Unity3D, Flash 3D pseudo engines, even Director 3D still are the top choices for games, apps, and interactives that need effects and possibly hardware rendering. But it is interesting that Google is essentially re-entering this debate after ditching on Lively and they must see some benefit to having a discussion about 3d on the web and 3d standards in general. I know they have lots of models and tools with SketchUp and Google 3D warehouse so who knows maybe they will take it over by being standards, open and information based.
What is O3D?
O3D is an open-source web API for creating rich, interactive 3D applications in the browser. This API is shared at an early stage as part of a conversation with the broader developer community about establishing an open web standard for 3D graphics.
Get involved
Download the plug-in (Windows and Mac) and explore the samples to see O3D’s capabilities. Linux users, see these instructions.
One thing is for sure, 3d development is still old school proprietary lock in in most cases. Working with 3d and tools like Maya, 3dsmax and others they have always been very non standard. From file formats to interfaces to even basic movements, all different. The general maths of 3d are the same and so should 3d pipelines. Formats like COLLADA are nice because they are starting to open up 3d pipelines and content creation but COLLADA still has many porting issues. FBX file format is another that is really useful and common making pipelines in Unity 3D, for instance, very nice. But it is owned and run by Autodesk who owns all the 3d apps (Maya, 3dsmax, SGI) and I am a bit leary of that method. But in the end 3d pipelines and rendering will be somewhat standardized and maybe the web will be hardware rendered one day. In most cases it is not needed, but for gaming, immersion, demos and other entertainment it could benefit heavily from a more standardized 3d pipeline and methods.
Here is a video with more information on Silverlight 3 Beta features that are matches of the latest Flash killer features in pixel shaders, 3d planes (ability to create pseudo-3d engines like papervision3d), local saving, pixel operations/bitmap handling, local messaging (silverlight to silverlight – like localconnection), out of browser desktop running ability of SL3, SEO and search indexing capabilities / deep linking navigation and more.
Video of the features of SL3 Beta, Demos, at a Slow Pace from #mix09
A few points after the video and taking a tour of the features.
The pixel shaders are written in HLSL (shader 2), however they are compiled to byte code and do not currently use the gpu for rendering. While the pixel shaders are very cool and the language to write them is standard pretty much for shaders in HLSL shader model 2 DirectX-based (the other is GLSL OpenGL based) they have not allowed this byte code to run on the GPU… yet. Here Flash and Pixel Bender actually are ahead there.
Although there are 3d planes which is very exciting, no good pseudo 3d engine exists yet matching the 3 in flash (papervision3d, away3d, sandy). When SL3 comes out I am sure we will see a few emerge or build them ourselves because this iteration of SL3 looks pretty fun.
Pixel based operations will be a huge advancement much like it was in earlier flash versions as it adds some demo scene type abilities and experiments with pixels that are fun. This also lends to doing cool things like shaders, effects, AR, face recognition, motion detection etc.
Effects like Blur and Drop shadow are good and the ability to add custom ones, great. However currently they are pretty performance intensive. They are also in Flash but there needs to be some refinement in SL3 effects before launch.
Desktop runnable apps in out of browser will be nice and this is a direct compete with Adobe AIR which was a surprise.
Local Communication supports desktop to browser communication.
Isolated storage (similar to shared objects) supports 1MB in browser, 25MB out of browser defaults.
This version of silverlight is really a 1.0 version as typical with most software. Version 2 or version 3 is usually what the initial design goals pan out. Much like the latest unity3d version (2.5) that has windows support and the iPhone SDK 3.0 that both came out this week, even actionscript 3 compared to earlier versions, these toolkits are finally iterated enough that they are really solid platforms for building cool stuff on and become platforms. The next version of all these could be very, very dangerous.
Huge missing features:
Although there are some great features in SL3 beta, it is still not done and it is still missing some key components that Flash has which make it very attractive in the interactive space.
Camera and Microphone support - Macromedia hired one of the smartest dudes around in Jeremy Allaire back in flash 6 days to help add support for Flash Communication Server (Flash Media Server now) Camer and Microphone support. One of the best R&D periods at Macromedia. SL needs this soon.
Printing support – what was long a problem in Flash is so in SL, there is no good printing support
No GPU usage for Pixel Effects/Shaders – (neither flash nor silverlight support hardware accelerated shaders in PixelEffects/Pixelbender – Pixel Effects/Shaders need GPU support (see Kevin Goldsmith’s article on GPU mixed with CPU and how this may or may not be good. However processors are speeding up and multi-core helps software rendering, the quality of GPU is well beyond what software rendering can deliver for a few years to come at least while architecture advances, probably more like 5-10 years.
No UDP plans yet - Adobe has RTMFP, SL sockets has no public plans for adding UDP that I have seen
No Alpha Channel in Video - You can do this with a shader though but not supported by default.
Silverlight 3 Video
Flash has the upperhand in video and probably will still even though SL3 has H.264. Flash added this at the same time and though they still have FLV which revolutionized web video they are now much broader in support in video than SL3. Silverlight has H.264 and VC1 support (their own FLV like codec). Still pretty cool a couple years ago there was no HD on the web now everyone has it in H.264 video support.
Currently nothing innovative, mainly catchup still, but here are some options
Silverlight 3 beta and the video below the features and highlights will look very similar to flash and flash community advancements over the last couple years. There is no innovation just yet.
But where that could happen is in socket support with UDP. Flash has moved on this in RTMFP and the beginning of larger scale networking support with UDP with samples like stratus. This is a huge differentiating feature for what I think will be game changer on the web (it already is on desktop mmos) in real-time or closer to real-time support for larger sets of users in online games like MMOs or virtual communities, even tools to make request based real-time sites like micro-blogging faster and able to handle more users (right now it is very linear if users get many followers, UDP will allow a better distributed framework for messaging).
Local Storage
Silverlight and Unity3D all need this, Flash could use better support for this. Local saving of a files for cache beyond the internet cache and greater than the 1MB/25MB limits of SL3 IsolatedStorage. This is an issue when you are making large scale games in that you need to save lots of assets to a client but to make it economical you want ot save more than the default internet cache amount. Flash Shared Object (Local) allow you to do this somewhat but it would be great to have a way to just download files for cache (upon user agreement) to store assets in bulk of allowable types (images, video, models, bundles) to the file system.
Hardware rendering for 3d support and UDP support will put Flash and SL3 on par with the killer Unity3D kit for making online web games and other activex/plugins like instantaction that allow you to do these things already.
The one thing SL has over Flash
Flash and Flex are great. But there is this massive division in the community and marketing of Flash. Silverlight is entirely unified and this has much to do with starting clean at a time that interactive development is heading more into a technology and developers control. Flash and Flex need to bring it together. AS3 has been out long enough that the people with skills have hopped on and taken it to a new level, mainly from programmers. If Adobe created a version of Flash that was a new IDE and consolidated Flex and Flash into just Flash, made the IDE as powerful as FDT or FlashDevelop3 there could be hope to bring the platform together. I understand they had to work it in slowly because it was a designers platform really (even though coders still pushed the limits in games and apps built on it) so they had to tip toe carefully on this to not alienate people. But now I think the division is a serious problem with the platform and must be addressed, noone expected Silverlight to be this quick on at least SL3 features. And even though the initial approach might have been bad as SL1 was a huge letdown, Microsoft does not give up and you can see in the XBOX360 and DirectX that they are very pursuant. DirectX really didn’t become huge until version 7 so these guys won’t relent.
I am not a huge fan of using the proprietary tools. Even in Flash I use as much open source as I can even though the player is locked, but Moonlight is something that trails Silverlight development and is a very unique thing in both open source and cross platform/multiplatform development. It is a clear relationship and aims to make Silverlight run on multiplatform mono including Linux. This could win out in the end who knows.
I have been really busy this week just delving into all them and hope to start making more cool and useful projects in them. The best part is right now is great to be an interactive or game developer as all major software companies and markets are focused on retaining good developers. I don’t’ recall a time other than the beginning the the web virtual land rush that has so many options and markets that skilled developers and designers can choose from. Good times.
Of course along the way making many improvements and showing great information on how and why the haXe version is faster which mainly boils down to a more strict virtual machine but flexible still with generics.
haXe is fast because it is a very highly optimized virtual machine language with compiler (and could be called a virtual machine to target other VMs similar to LLVM with the ability to target the Neko VM, AVM2 or Javascript, it is more than just a language) by Nicolas Cannasse that may one day overtake directly coding for the AVM2 or maybe we will even see haXe have more influence on flash soon for performance gains. Some of the Alchemy LLVM virtual machine work is similar in nature to what haXe does and helps the language become an abstraction and translates into highly optimized code from very powerful and productive language syntax.
Anyways, I ramble, be sure to check out Data Structures for Game Developers by Polygonal Labs now ported for haXe as hx3ds if you are doing any sort of work in AS3 or haXe for AS3 it will be worth your while and provide a very common and useful data structures capabilities into your production that is highly optimized from one of the best AS3 developers.
As the name suggests, hx3ds is a port of as3ds for haXe and is now available at lib.haxe.org. hx3ds only supports the flash player 10 target, as it makes extensive use of the Vector class. If you need data structures that compile across all platforms, take a look at colhx instead.
Here’s a list of new features:
orders of magnitude faster
collections now support clone() and shuffle() operations
object pooling framework
revised graph, tree and linked list classes
memory manager for the virtual memory API (more on this soon)
The Structures Included
Multi-Dimensional Arrays
The library contains a two-dimensional and three-dimensional array. They are both implemented by a single linear array rather than nested arrays. This is the fastest method in flash to simulate multi-dimensional arrays and outperforms the nested array method because multiple array lookups are slower compared to one lookup combined with a simple arithmetic expression (which you can also often precompute in the outer loop). The most obvious application would be a tilemap in 2d or a layered tilemap in 3d.
Queue
This is also called a FIFO structure (First In – First Out). The queue comes in two variations, which have the same methods, but differ in their implementations: There is the arrayed queue, which obviously uses an array internally, and the linked queue, which is build upon a linked list. They are both very similar, except that the arrayed version has a fixed size and is faster.
A common application would be a command queue – imagine you have a unit in a strategy game and apply many commands which the unit should follow. All commands are enqueued and afterwards dequeued and processed in order.
Stack
Also commonly know as a FILO structure (First In – Last Out). Like the queue, this comes in two flavors: arrayed and linked. A stack is often not used directly, but a very important concept in programming. Please note, that a queue and a stack are not real structures, because they just define how data is accessed rather then stored.
Tree
A node-based structure. Every tree starts from a single node, called the root node. The root node can contain any number of child nodes, and every child node can again contain children. A tree node with no children is called a leaf node. In fact if you draw the nodes of a tree it looking like a real tree with branches. The AS3 display architecture is also a tree structure, so you could use this to manage your display objects and update them by traversing through the tree. Also, this is useful for decision trees, BVHs, storing a plot line or storing data recursively by applying the composite pattern.
Binary Tree
This is just a specialized kind of tree where each node is only allowed to have up to two children, called the left and right node. Binary trees are very often used for parsing input data, for example arithmetic expressions or when building a scripting system.
Binary Search Tree (BST) and Hash Table
Both structures store data that can be retrieved quickly by using a key. The method however differers greatly: The BST uses a recursive approach to split up large amounts of data into smaller sets. A hash table stores sparse key-based data using a hash-key in a small amount of space.
Linked Lists
A linked list is similar to an array. The main difference is that in an array, each cell contains just the data and is accessed by an index. A linked list consists of several node objects, which in addition to storing the data, manage a reference to the next node (singly linked) or to the next and previous node (doubly linked) in the list. Think of it as a more natural approach to work with sequential data.
Other benefits are that you can insert and remove data quickly by just calling the appropriate method on the node itself – you don’t have to manage array indexes. Also in AS3 object access is faster than array access, so it competes very well in terms of performance when iterating over the list.
Heap and Priority Queue
A Heap is a special kind of binary tree in which every node is bigger than its child nodes. Whatever you throw into a heap, it’s automatically sorted so the item with the ‘most significant’ value (depending on the comparison function) is always the front item. A priority queue is build upon the heap structure, and can manage prioritized data – which can be used in limitless ways.
Graph
A graph is a loose node-based structure. Nodes are connected with arcs, and every node can point to any other node. They can also point to each other creating a bi-directional connection. It is essential for path finding, AI, soft-body dynamics with mass-springs systems and a lot more.
Bit Vector
A bit vector is some kind of array in which you can store boolean values (true/false – 1/0) as close as possible without wasting memory. I currently can’t think of a reasonable application, because usually you should have enough memory – but it’s nice to have because it shows basic bit masking operations.
Adobe will essentially open up the RTMP protocol officially. RTMP has been used in other tools such as Red5 and haXe video for some time now. But officially having it open will make it possible for more products built on it. I am sure that most of this is to combat silverlight and to gain more video users that can play flash formats. RTMP spec will be posted here when ready.
RTMP provides an enhanced and efficient way to deliver rich content. Developers and companies will have free and open access to the documented RTMP specification to help enable unparalleled delivery of video, audio and data in the open AMF, SWF, FLV and F4V formats compatible with Adobe Flash Player.
Adobe has also been working on more real-time protocol tools based on UDP instead of TCP (which RTMP is based) that fall under RTMFP using ordered UDP that will be interesting to watch evolve. Stratus is so far a sample of what is to come there.The UDP based real-time tools will be able to beat the capabilities of TCP based real-time tools when using authoritative servers.
But with the RTMP announcement, multiuser and video applications should thrive even more with an open RTMP spec.
Want to build a video chat application, multi-player games or voice-over-ip applications for the Flash Player or AIR without worrying about setting up a server infrastructure? Stratus (which we showcased at MAX) is your new best friend.
Stratus is a beta hosted rendezvous service that helps establish communication between Flash Player or AIR clients. Once two clients are connected to Stratus, they can send data directly client to client. The APIs in Flash Player 10 and Adobe AIR 1.5 allow for point-to-point communication between a small number of subscribers. Publishers have to send data to all subscribing clients, so the number of subscribers is limited to the available bandwidth on the publisher end.
This must be one of the first Real-Time Media Flow Protocol (RTMFP) protocol usage programs from Adobe? Basically this protocol is adding better UDP or broadcast support which allows for larger sets of users and is common in large scale real-time games. Here it seems to be more of a peer to peer usage rather than authoratative approach (maybe flash media server will have large user set support with this) which limits to about 15 users or the lowest latency in the group with anything close to real-time syncing. Stratus seems like more of a matchmaking middle man to help with nat punchthrough and then it relies on peer to peer. But more fun in store checking it out.
P2P like this can’t really be used for games due to cheating unless one client is the independent server but this works great for small file sharing apps, whiteboards, chats, watching videos at the same time, etc.
The Open Screen Project will address potential technology fragmentation by enabling the runtime technology to be updated seamlessly over the air on mobile devices. The consistent runtime environment is intended to provide optimal performance across a variety of operating systems and devices, and ultimately provide the best experience to consumers.
To support this mission, and as part of Adobe’s ongoing commitment to enable Web innovation, Adobe will continue to open access to Adobe Flash technology, accelerating the deployment of content and rich Internet applications (RIAs). This work will include:
Removing restrictions on use of the SWF and FLV/F4V specifications
Publishing the device porting layer APIs for Adobe Flash Player
Publishing the Adobe Flash® Cast™ protocol and the AMF protocol for robust data services
Removing licensing fees – making next major releases of Adobe Flash Player and Adobe AIR for devices free
This is big. It has mutiple prongs. Adobe would like to make Flash a common mobile standard, hrm no Apple on this list (it is probably a slight move there against that).
Adobe would also like to continue their lead of web video. And they finally are recognizing the closed format of SWF is not as desirable as an open one, there is still considerable control for Adobe over the player. But they get insights and contributions from many large companies to help make it work on their platform, engraining the format further. The providers, especially mobile and internet tv, will want to provide good user experiences to compete with the iPhone and regular T.V. respectively. Flash being open helps both of those markets.
Adobe is also moving further to open source key formats and technologies like the recent Flex 3 SDK and now the AMF format which was a roadblock. This is probably good news for servers like Red5 and also many other media servers and remoting services in many more place. AMF is particularly nice because it is a binary, extremely compact and limited bloat format. Without it being open it loses much of its benefit as a standard. Being open and a further crunching from the XML bloat services, this can be very good for many reasons such as throughput and faster services, apps and games with remote data.
Another reason is the desktop market. They want Flash to work flawlessly on Linux but they don’t’ have the manpower for a 2% market share. So this is a very smart move for the desktop players (AIR, and Linux Flash).
The only thing that partially makes it scary is the line up, Sony, Verizon, etc. As long as these are contributors and not partners in DRM crime then we have something. Hopefully they are in it to make better entertainment and mobile platforms cheaper.
Being able to peer into the code and a move to allow better open integration makes it a better platform, where better stuff can be built on top of that. Let’s hope it is done right. Everyone is making Apple look really closed and locked down lately.
RT @waxpraxis: Love the term "incidental complexity" - complexity not inherent to a problem, but from the way the solution is derived. 22 hrs ago
boo! iMac having hard drive issues only 2 weeks after upgrading to snow leopard, hopefully DiskWarrior can save it. pi day has been tainted. 2 days ago
Alice in Wonderland was pretty good, great visuals, pretty decent dose of Burton. 3 days ago
OpenGL 4.0 released, in a couple years we will have opengl 4 enabled cards w tessellation... http://bit.ly/cggF3f - puts it on par with DX11 5 days ago
Finally got an openssl network lib running pc, mac, iphone + nix..biggest problem was iphone os compared to iphone sim (arm vs i386 targets) 5 days ago
Triangle was pretty decent for a netflix movie, totally ripped the premise from Timecrimes (which was really good) but still interesting. 6 days ago
So many cool and useful technologies are unveiled at SIGGRAPH every year, this year at SIGGRAPH 2009 was no different. Khronos Group, behind the new guidance of OpenGL, OpenGL ES, OpenCL, OpenVG, COLLADA etc, came another big announcement about hardware rendering within the browser. WebGL is now an official standard being developed at Khronos Group to bring javascript control of OpenGL to browsers… Wow!Khronos Details WebGL Initiative to Bring Hardware-Accelerated 3D Graphics to the Internet
