Archive for the ‘DEVELOPMENT’ Category
Chromium is moving to GPU hardware accelerate rendering all types of web content as much as possibly with their latest efforts.
For some time now, there’s been a lot of work going on to overhaul Chromium’s graphics system. New APIs and markup like WebGL and 3D CSS transforms are a major motivation for this work, but it also lets Chromium begin to take advantage of the GPU to speed up its entire drawing model, including many common 2D operations such as compositing and image scaling. As a lot of that work has been landing in tip-of-tree Chromium lately, we figured it was time for a primer.
The primer they are looking at is not just rendering the content made in WebGL, CSS3 3d transformations and more but the entire final pass of the output. This leads to some very interesting years ahead in browsers. With Chromium, IE9, Firefox and Safari all now with aspects of hardware rendering and acceleration via the GPU, anyone not doing GPU acceleration is seemingly behind the curve that seemed to start in 2007ish to a culmination of today’s latest browsers.
After these layers are rendered, there’s still a crucial last step to blend them all onto a single page as quickly as possible. Performing this last step on the CPU would have erased most of the performance gains achieved by accelerating individual layers, so Chromium now composites layers on the GPU when run with the –enable-accelerated-compositing flag.
Web content will get really interesting over the next couple years. Even basic computers now have a GPU and bottom of 32MB video memory. Why aren’t we using those GPUs as much as possible for web content and web games. The time of software rendering might be coming to an end now that processors seem to have topped out and the bottom level computer is capable of handling a decent amount of video memory. It will be easier to justify useful graphics acceleration with a better user experience when we can take advantage of all the computer/device has to offer.
Microsoft has released Silverlight 3 to the web a day earlier than expected. You can go ahead and grab Silverlight 3 RTW build 3.0.40624.0 (4.69MB) from Microsoft.com/Silverlight. Version 3 supports Windows Internet Explorer 6/7/8, Firefox 2/3, and Safari 3/4. In addition, the Silverlight 3 SDK (9.5MB) and the Silverlight 3 Tools (32.2MB) have been posted on the Microsoft Download Center. (arstechnica)
Granted there are lots of years of gain that Flash has on Silverlight but the path that Silverlight it following leads right to Flash. Hopefully this will lead to more innovation on both sides (they might need it with Google Wave pushing html5 <canvas>).
One very nice element of Silverlight since it has been released at verison 1.0 is the HD video support. It has gotten better with each release. This release has smooth streaming support that is pretty impressive for web video.
Interestingly they chose the open source Blender Foundation project Bug Buck Bunny to demonstrate the smooth streaming feature. Ryan Rea has a bit of analysis on how well the video plays across a quad core and memory compared to flash hd video.
Roy Schestowitz plays the flip side and calls this a ‘silver-lie’ released and has lots to say about the Silverlight 3 release and even using Big Buck Bunny to demo it. He states a true fact that up til now lots of companies have abandoned Silverlight in favor of Flash (mlb, nyt etc). It is still used at Netflix but that has an XBOX deal to play Netflixon xbox360.
Flash didn’t really get good until version 4. Typically software is a real version at version 3, that is when most software has the goals and ambitions of 1.0 fully complete and integrated. We shall see how things play out but I still think Silverlight has a long way to go in winning over developers, myself included, but competition is never a bad thing when you are wanting to see innovation.
AIR is very popular for creating twitter clients, Sönke Rohde just made it much easier to make AIR apps for twitter with an AS3 library for Twitter. This library is built on top of core oauth as3 library by iotashin. Core OAuth as3 library is a standard OAuth library this can be used for your own OAuth backends or connecting to other OAuth services as well.
A very nice feature of this library, in addition to being coded cleanly and as3 style, is the ability to have the Twitter OAuth page render inside of Flash.
Instead of opening the Twitter authorization page in the browser the library also contains OAuthLoader which is a wrapper around HTMLLoader which enables to directly show the authorization page within an AIR window:
// use this in the requestTokenHandler instead of navigateToURL var loader:OAuthLoader = new OAuthLoader(); loader.load(request); loader.percentWidth = 100; loader.percentHeight = 100; var w:Window = new Window(); w.width = 800; w.height = 400; w.title = req.url; w.addChild(loader); w.open();
Flash 10 will be ready for mainstream hopefully by the end of this year, or early ’10 when the penetration numbers will be up in or around the 90% range via zeh fernando based on previous trajectories.
With that, Flash 10 has many great new things such as the Vector structure that allows a collection of a certain type, which results in a faster collection because of the known type. So anywhere where Arrays are used, that is a possible candidate for a performance increase within some code because you are asking the virtual machine to do less work on each loop (not having to dynamically find out the type).
ByteArray (Thibault Imbert) has demonstrated that for the JPEG encoding in corelib it is up to 2.5 times faster using Vectors than Arrays. Your mileage may vary heavily but it is almost a guaranteed speed boost due to less work. This obviously has great possibilities for speeding up code that uses lots of arrays.
Due to the performance boost the Vector does have some constraints in the typical give and take of coder flexibility with compiler and virtual machine overhead. Vectors are more explicit and strongly typed which is why they are fast, but this is also limiting.
In addition to the data type restriction, the Vector class has other restrictions that distinguish it from the Array class:
- A Vector is a dense array. Unlike an Array, which may have values in indices 0 and 7 even if there are no values in positions 1 through 6, a Vector must have a value (or
null) in each index.
- A Vector can optionally be fixed-length, meaning the number of elements it contains can’t change.
- Access to a Vector’s elements is bounds-checked. You can never read a value from an index greater than the final element (
length - 1). You can never set a value with an index more than one beyond the current final index (in other words, you can only set a value at an existing index or at index
[ Vector docs ]
ByteArray not only used Vectors heavily but did other optimizations that are always good to do, even though optimization is evil when you are working with precious client side resources ensuring an optimized base starting point can be a good thing.
So what did I do ?
- I used bitwise operators as much as possible.
- I replaced all Arrays with fixed length Vectors.
- I used pre-incrementation rather than post-incrementation (thanks Joa for this one ;)).
- I casted to int all my Vector indices access.
- Other minor stuff you always do to optimize your code
Other sources as well for even more optimization or shall I say efficient AS3:
- Polygonal Labs Using Object Pools
- Joa Ebert Tweening and Object Pools
- Joa Ebert Optmization Wiki
- Sean Moore Efficient AS3 roundup (great list of resources for more at the bottom)
- Lost in Actionscript Efficient AS3 Tips
- Big SpaceShip Mouse Leave Tip (flash can hog resources so when it isn’t in focus then let is stop tasking the processor)
- Grant Skinner Idle CPU in AIR FPS Tip (set fps to a lower amount when it is out of focus)
pyamf is pretty sweet for Flash remoting with Pythonic server side, but now we have two nicely done and integrated remoting kits for python on the server side.
amfast is a new remoting library that looks to be as sweet as pyamf (where sweet == fast and useful). I am checking out amfast now but the speed boost alone might be worth it. For instance, working with real-time games, when you need static content you need to grab that quickly sometimes via a content service. The faster that link the better. It also has Twisted integration which is great for networking and SQLAlchemy integration which is in my opinion the best ORM for python (pyamf has twisted, django, pylons, sqlalchemy as well)
amfast is well documented and has some great examples. If you have the Python addiction, check it.
- AmFast is a Flash remoting framework for Python.
- AmFast can use AMF to communicate between Python and Flash, Flex, and any other system that supports AMF.
- AMF is a binary object serialization protocol used by Actionscript based applications.
- Support for NetConnection and RemoteObject RPC.
- Support for Producer/Consumer ‘push’ messaging with HTTP polling, HTTP long-polling, and real-time HTTP streaming channels.
- Support for authentication with NetConnection and RemoteObject.
- Flexible Target mapping system to map message destinations to invokable Target objects.
- Support for ChannelSets with multiple Channels to expose resources in different ways.
- Built in Channels for CherryPy, Twisted Web, and plain WSGI.
- Support for configurable Endpoints. Use AmFast’s built-in AMF encoder/decoder C-extension, or use an external AMF encoder/decoder, such as PyAmf for a pure-Python implementation.
AMF Encoder/Decoder Features
- AMF0/AMF3 encoder/decoder written in C as a Python extension for speed.
- More than 10x faster than the PyAmf encoder/decoder (even when using PyAmf’s optional C-extension).
- Map custom classes with ClassDef objects for complete control over serialization/de-serialization.
- Full support for IExternalizable objects.
- Data persistence with SqlAlchemy including remotely-loadable lazy-loaded attributes.
- Actionscript code generation from ClassDef objects.
Director 11.5 was quietly released last week at GDC with a few nice upgrades.
- The sound library is updated to Dolby surround 5.1.
- Director 11 now supports ByteArray and binary data handling.
- It also states support for Flash 9 swfs. Previously Director 11 did not work well/atall with AS3/Flash 9 swfs which made it nearly useless.
- Streaming support for audio and video with RTMP (red5, flash media server, etc)
- Updated video support
- Bitmap and audio filters for video
I still think Director is on decline unless they open up the development platform, lose Lingo and allow a real IDE to develop with. So frustrating being restrained to that IDE that is not very flexible and cumbersome to extend and code in when you compare it with cutting edge IDEs like Unity3D or open source flash IDEs like FlashDevelop. It has been completely removed from our workflow for some time due to new Flash 2.5D engines such as papervision 3d, away 3d and sandy or for more immersive hardware rendered 3d, unity3d.
|Adobe Director version comparison chart|
|Product features||Director 11.5||Director 11||Director MX 2004|
|Support for 5.1 surround sound||Yes||No||No|
|Real-time audio mixing||Yes||No||No|
|Audio effects and DSP filters||Yes||No||No|
|H.264 MPEG-4, FLV, and F4V video support||Yes||No||No|
|Streaming support for audio and video with RTMP||Yes||No||No|
|Ability to apply audio filters on a video||Yes||No||No|
|Ability to apply bitmap filters on a video||Yes||No||No|
|Google SketchUp file import||Yes||No||No|
|Enhanced physics engine with support for dynamic concave rigid bodies||Yes||No||No|
|ByteArray datatype for binary data handling||Yes||No||No|
|Multiple undo/redo for text editors||Yes||No||No|
|Text rendering and performance optimization||Yes||No||No|
|Cross-domain policy support for Adobe Shockwave® Player||Yes||No||No|
|Mac OS X Leopard support||Yes||No||No|
|Microsoft DirectX 9 support||Yes||Yes||No|
|Advanced physics engine with included NVIDIA® PhysX™ support||Yes||Yes||No|
|Microsoft® Windows Vista® support||Yes||Yes||No|
|Support for Intel® based Macs||Yes||Yes||No|
|Cross-platform projector publishing||Yes||Yes||Yes|
|Web publishing with Adobe Shockwave Player||Yes||Yes||Yes|
|Support for more than 40 video, audio, and image file formats, including SWF||Yes||Yes||Yes|
Here is a video with more information on Silverlight 3 Beta features that are matches of the latest Flash killer features in pixel shaders, 3d planes (ability to create pseudo-3d engines like papervision3d), local saving, pixel operations/bitmap handling, local messaging (silverlight to silverlight – like localconnection), out of browser desktop running ability of SL3, SEO and search indexing capabilities / deep linking navigation and more.
Video of the features of SL3 Beta, Demos, at a Slow Pace from #mix09
A few points after the video and taking a tour of the features.
The pixel shaders are written in HLSL (shader 2), however they are compiled to byte code and do not currently use the gpu for rendering. While the pixel shaders are very cool and the language to write them is standard pretty much for shaders in HLSL shader model 2 DirectX-based (the other is GLSL OpenGL based) they have not allowed this byte code to run on the GPU… yet. Here Flash and Pixel Bender actually are ahead there.
Although there are 3d planes which is very exciting, no good pseudo 3d engine exists yet matching the 3 in flash (papervision3d, away3d, sandy). When SL3 comes out I am sure we will see a few emerge or build them ourselves because this iteration of SL3 looks pretty fun.
Pixel based operations will be a huge advancement much like it was in earlier flash versions as it adds some demo scene type abilities and experiments with pixels that are fun. This also lends to doing cool things like shaders, effects, AR, face recognition, motion detection etc.
Effects like Blur and Drop shadow are good and the ability to add custom ones, great. However currently they are pretty performance intensive. They are also in Flash but there needs to be some refinement in SL3 effects before launch.
Desktop runnable apps in out of browser will be nice and this is a direct compete with Adobe AIR which was a surprise.
Local Communication supports desktop to browser communication.
Isolated storage (similar to shared objects) supports 1MB in browser, 25MB out of browser defaults.
This version of silverlight is really a 1.0 version as typical with most software. Version 2 or version 3 is usually what the initial design goals pan out. Much like the latest unity3d version (2.5) that has windows support and the iPhone SDK 3.0 that both came out this week, even actionscript 3 compared to earlier versions, these toolkits are finally iterated enough that they are really solid platforms for building cool stuff on and become platforms. The next version of all these could be very, very dangerous.
Huge missing features:
Although there are some great features in SL3 beta, it is still not done and it is still missing some key components that Flash has which make it very attractive in the interactive space.
- Camera and Microphone support - Macromedia hired one of the smartest dudes around in Jeremy Allaire back in flash 6 days to help add support for Flash Communication Server (Flash Media Server now) Camer and Microphone support. One of the best R&D periods at Macromedia. SL needs this soon.
- Printing support – what was long a problem in Flash is so in SL, there is no good printing support
No GPU usage for Pixel Effects/Shaders– (neither flash nor silverlight support hardware accelerated shaders in PixelEffects/Pixelbender – Pixel Effects/Shaders need GPU support (see Kevin Goldsmith’s article on GPU mixed with CPU and how this may or may not be good. However processors are speeding up and multi-core helps software rendering, the quality of GPU is well beyond what software rendering can deliver for a few years to come at least while architecture advances, probably more like 5-10 years.
- No UDP plans yet - Adobe has RTMFP, SL sockets has no public plans for adding UDP that I have seen
- No Alpha Channel in Video - You can do this with a shader though but not supported by default.
Silverlight 3 Video
Flash has the upperhand in video and probably will still even though SL3 has H.264. Flash added this at the same time and though they still have FLV which revolutionized web video they are now much broader in support in video than SL3. Silverlight has H.264 and VC1 support (their own FLV like codec). Still pretty cool a couple years ago there was no HD on the web now everyone has it in H.264 video support.
Currently nothing innovative, mainly catchup still, but here are some options
Silverlight 3 beta and the video below the features and highlights will look very similar to flash and flash community advancements over the last couple years. There is no innovation just yet.
But where that could happen is in socket support with UDP. Flash has moved on this in RTMFP and the beginning of larger scale networking support with UDP with samples like stratus. This is a huge differentiating feature for what I think will be game changer on the web (it already is on desktop mmos) in real-time or closer to real-time support for larger sets of users in online games like MMOs or virtual communities, even tools to make request based real-time sites like micro-blogging faster and able to handle more users (right now it is very linear if users get many followers, UDP will allow a better distributed framework for messaging).
Silverlight and Unity3D all need this, Flash could use better support for this. Local saving of a files for cache beyond the internet cache and greater than the 1MB/25MB limits of SL3 IsolatedStorage. This is an issue when you are making large scale games in that you need to save lots of assets to a client but to make it economical you want ot save more than the default internet cache amount. Flash Shared Object (Local) allow you to do this somewhat but it would be great to have a way to just download files for cache (upon user agreement) to store assets in bulk of allowable types (images, video, models, bundles) to the file system.
Hardware rendering for 3d support and UDP support will put Flash and SL3 on par with the killer Unity3D kit for making online web games and other activex/plugins like instantaction that allow you to do these things already.
The one thing SL has over Flash
Flash and Flex are great. But there is this massive division in the community and marketing of Flash. Silverlight is entirely unified and this has much to do with starting clean at a time that interactive development is heading more into a technology and developers control. Flash and Flex need to bring it together. AS3 has been out long enough that the people with skills have hopped on and taken it to a new level, mainly from programmers. If Adobe created a version of Flash that was a new IDE and consolidated Flex and Flash into just Flash, made the IDE as powerful as FDT or FlashDevelop3 there could be hope to bring the platform together. I understand they had to work it in slowly because it was a designers platform really (even though coders still pushed the limits in games and apps built on it) so they had to tip toe carefully on this to not alienate people. But now I think the division is a serious problem with the platform and must be addressed, noone expected Silverlight to be this quick on at least SL3 features. And even though the initial approach might have been bad as SL1 was a huge letdown, Microsoft does not give up and you can see in the XBOX360 and DirectX that they are very pursuant. DirectX really didn’t become huge until version 7 so these guys won’t relent.
I am not a huge fan of using the proprietary tools. Even in Flash I use as much open source as I can even though the player is locked, but Moonlight is something that trails Silverlight development and is a very unique thing in both open source and cross platform/multiplatform development. It is a clear relationship and aims to make Silverlight run on multiplatform mono including Linux. This could win out in the end who knows.
Great iterations of software happened this week in the latest unity3d version (2.5) that has windows support and the iPhone SDK 3.0 and now SL3 is quite a surprise in feature set.
I have been really busy this week just delving into all them and hope to start making more cool and useful projects in them. The best part is right now is great to be an interactive or game developer as all major software companies and markets are focused on retaining good developers. I don’t’ recall a time other than the beginning the the web virtual land rush that has so many options and markets that skilled developers and designers can choose from. Good times.
Alchemy is going to shake things up a bit. As witnessed before from Quake running in flash and now ODE compiled to run in flash using Alchemy (LLVM based). It is an early test but shows what could be possible.
Mihai Pricope has a post with sources on how he got the ODE (Open Dynamics Engine) a great open source physics engine for 3D, running on the AVM2 Flash Player virtual machine.
I’ve took Alchemy for a test and decided to compile ODE (Open Dynamic Engine). Just to add yet another physics engine to the Flash World. It was a hell of a ride but I finally got to produce some bouncing balls :). For a still unknown reason some as 3d libraries have been very slow to render 6 translucent walls and 2 balls. Papervision3D seems to move quite decent.
Flash 10 will become mainstream shortly and with that the possibilities of using Alchemy in your projects is becoming a reality for production. But what specifically can you do with Alchemy, a project that helps to compile C/C++ code into AVM2 capable files?
Alchemy described from Adobe:
With Alchemy, Web application developers can now reuse hundreds of millions of lines of existing open source C and C++ client or server-side code on the Flash Platform. Alchemy brings the power of high performance C and C++ libraries to Web applications with minimal degradation on AVM2. The C/C++ code is compiled to ActionScript 3.0 as a SWF or SWC that runs on Adobe Flash Player 10 or Adobe AIR 1.5.
Alchemy is based on the LLVM Low Level Virtual Machine that allows new levels of code translation. Maybe this can lead to more effective and performing code to run on the iPhone with flash player 10. Or some type of system that allows flash developers to code in AS3 or take projects and get them ready to run on the iPhone much like some of the Java to Cocoa compilation systems and Unity3D using mono to compile down to iPhone capable code.
The guys over at Flashbang Studios, a web game development studio in phoenix, az, and some of the most visible developers in the Unity3d space with Blurst, released something that may interest both FlashDevelop users and Unity3d developers.
FlashDevelop, is by far the best Flash / Flex /haXe IDE in my opinion so it is really great to release this for Unity. I hope one day I or someone has the time to port to Mono so it can be used on Macs even with the 140 pinvokes, it would be a good spread mechanism for mono.
With Unity3d coming to windows soon, UnityDevelop could be a good go to IDE for unity if you aren’t using solely C# with VS.NET. Currently this is based on FlashDevelop2 source code.
- Sample iPod iTouch Mesh data (meshdata.js) – contains models
- Code from the main engine (parpevision.js)
- Sample Viewport and main app (touch.js from demo 3)
Here is the code for the parpevision.js file and the mini engine, it is an MIT license. (more…)