January 31^st/ Manifesto of the UI

I’ve been thinking more and more lately about the state of user interface and it’s evolution path — it’s something that I can’t get out of my mind. Over the past few years (we’re talking less than 5), we’ve seen user interfaces across the digital world morph from a static experience into highly dynamic interactive experiences. Web sites like Facebook and MySpace have proven that interactivity and the ability to relate real world ideas to the digital realm wins over features and functionality. Applications like iTunes have shown that how data is presented and you interact with that data is more important than how your computer processes the same data.

As such, I’d like to pose a simple question to those front end developers out there: What do you think the future of UI technology will look like?

Where we are right now

Right now, we have a huge group of platforms to develop with. We’ve got platforms for the web, platforms for the desktop, and platforms for both. In the past few months, we’ve seen the desktop and the web reach out toward one another, and the line is really starting to blur.

Web

On the web side of things, HTML is the prevalent technology, with XML and Flash following closely behind. The web has seen a revolution is technologies that enable rich interfaces. We can deliver high definition video, manipulate XML through the browser, and change HTML & CSS on the fly reliably across all popular browsers.

Along with the rise of technologies that give us the ability to create rich interfaces has been a rise of rationality and code design. We’ve seen terms like the semantic web being thrown around as well as the rise of highly refined ideas like microformats. Developers have taken a relatively small orthogonal toolset — HTML, CSS, and Javascript — and refined it into an art.

Unfortunately the two movements have happened very much in parallel with little to no crossover. We have some extremely powerful technologies that allow us to create incredible rich experiences — but few rational and idealistic methodologies for creating rich user interfaces.

Desktop

In the past two years, the desktop has seen nothing short of a revolution in interface design. With the release of Windows Presentation Framework in Vista / .NET 3.0, and the rise of OS X technologies like Core Text, Core Image and Core Animation we’ve seen immense power become available to the user interface developer.

No longer does creating interactive 3D interfaces require a Ph.D in OpenGL — the power has been brought to the surface and now developers have great tools to create amazing interfaces with little effort through scripting languages and frameworks.

Where we are headed

Now for a little more proclamation, and a whole lot less supporting data. I’m excited for where we’re headed in UI development, and I think you should be too.

A semantic web

The semantic web goes much beyond building table-less layouts with CSS for styling. It goes to the core of the information we’re presenting. It means providing information in an open format for all to consume. Specifications like RSS, Atom, and even microformats like hCard and hReview provide standard templates for providing and consuming data.

We’ll see more services valuing their API more than their front-end. Twitter has orders of magnitude more traffic on it’s API than it does on it’s website. This is because they provide their data in semantic form.

I think we’ve seen the first revolution in the semantic web: RSS. The next step in my mind is seeing more and more JSON provisions. JSON is a logical next step: it’s lighter, simpler, more easily consumed, and even more flexible than XML. However with flexible back-ends like Rails, we’re seeing an almost transparent conversion between XML, JSON, and Enumerables — so it’s anybody’s guess if any format will even win out.

CSS will not evolve

I am confident that CSS is an excellent technology for typography, however after using Flex and WPF I am not convinced that it will ever translate into a full-on styling engine. Using real-time vectors, gradients, and animations is something that a styling engine needs to support: and it’s something that CSS simply is not made to handle.

Furthermore, any improvements made to the CSS spec from v2 on will not be adopted readily. Don’t get me wrong: portions of CSS3 will be adopted by individual browser makers and eventually they will be available on a large portion of people’s machines. But I do not believe that CSS3 in all its glory will be implemented and in production use within the next decade. By the time CSS3 is supported we’ll have moved past CSS3 and it’s limitations.

CSS support will be refined to the point of perfection for typography. One will be able to specify embedded fonts, anti-aliasing, kerning, leading, as well as performance on screen and in print through CSS. WebKit will be the first rendering engine to tie into OS-level typographical controls with pass-through attributes to Core Text. Soon afterwards Flash will refine the new text rendering in FP10 to be available through CSS. Windows developers will remain to have poor control over anti-aliasing for several years: despite having the best sub-pixel text rendering engine, ClearType has been abandoned since it’s inception and Microsoft shows no concern for designer’s pleas for typographic control.

Rise of the device

For so long web developers classified “devices” as small, hand held mobile devices with limited resolutions. But we were wrong. Today there are more Wii’s in the world than there are tablet PCs. We’re finding online-capable devices like the XBOX360, the Wii, the PS3, the iPhone, and even dashboard navigation devices in our cars pop up all over the world.

One thing I’ve learned is that we don’t know where the market will head with devices. All we know is that the resolutions game isn’t terribly applicable anymore. We must think further past resolutions, and think about input methods and connection speeds. These are the variables we should be designing for.

The device will be huge. It is huge. Deploying to one platform is a thing of the past — instead we will be targeting the platforms of our choice.

Should we spend time optimizing our website for IE6, or the Wii? This question isn’t too far off.

Object Oriented

The recent past has proven that the object-oriented pattern has not only suited itself perfectly for visual scripting, but become extremely popular among hobby developers (the ones who start trends). We’ve even seen prototype languages like Javascript beaten into submission until OO techniques could be implemented with frameworks like Prototype and MooTools.

I would say that weakly typed languages will win over strongly typed, but with the awesome success of Actionscript 3 and C^#, I can’t say that will be a reality. I do believe developers all of the world would lead happier lives with a weak typed ECMAScript based language, but I can’t see it happening in lieu of performance gains through strong typing. You can always improve the power of the back-end machine, but rarely the client’s machine (where visual scripting is executed).

Relevance to the real world

Early interface development revolved around allowing humans to access functionality — programs like Microsoft Office are the epitome of this design. The interface’s sole goal is to help you use the functionality behind the glossy buttons — to crunch numbers in excel or save hundreds of pages of formatted words into a printable format.

However, interfaces will start gravitating towards real-life connections. Interfaces will be built that help you communicate with Tom in Virginia, or help you bake that special German chocolate cake your mother used to make.

People have misjudged this shift towards interactivity as a shift towards user-generated content. This is simply not the case. Sites like Facebook and Twitter prove that people want to interact through technology, not with technology. Applications like iTunes have proved that people want to have a physical connection to real world objects through their UI.

Convergence of the “Desktop” and the “Web”

Internet connectivity has become status quo. The choice of whether to build a desktop application that connects to the internet — or an internet application that connects with the desktop will become a design choice, not a technology consideration. A small handful of technologies (like AIR, Prism) will tie in os-specific functionality into a common UI framework. Meanwhile, a small handful of specifications (like RSS, ActiveResource) will standardize data available through the internet.

The end result will be the same: while the syntax of the technologies will differ, their approach will be extremely similar. I’ve already seen this happen: developing an application with WPF (desktop technology) is almost identical to developing an application with Flex (web-based technology). It’s also fair to note that WebKit will most definitely be a part of this end result. The ubiquity of this rendering engine across Adobe’s AIR runtime, the iPhone, Windows, and OSX make this rendering engine a clear winner.

Share your opinion

I really want to hear people’s opinion on this. It’s something I’m very passionate about, and have thought about a lot. I want to hear arguments, disagreements and proclamations of what’s going to happen. No one ever got anywhere agreeing with everyone — let’s hear your side.

1. 

   Oliver

   February 1^st | #

   A couple of small points i’d make is that WebKit already supports CSS based animation (http://webkit.org/blog/138/css-animation/) so that is clearly feasible, and as far as vector graphics goes CSS is patently unsuited — CSS is for styling content, not content itself. For vector graphics you should look at SVG, which uses css for styling, and supports filters, gradients, vector art, media, animation, and scripting — all of which are in standard non-proprietary ways and it integrates into the DOM, which plugins like flash cannot do easily.

   When you look at AIR remember that you are looking a flash, with the added ability to view webpages, not some kind of extended html or web technology. Just flash with a pretty new dress.

2. 

   Kyle

   February 1^st | #

   Oliver: Saying the CSS-based animations available in WebKit are suitable as a fully functional animation engine is naive and simply untrue. Can you move objects? Can you scale them? Can you specify easing functions? Can you animate arbitrary values?

   When i refer to styling engines, I mean full on styling engines. Not a method for styling. If I could make a comparison, I would talk about creating artwork. CSS is a lot like acrylic paints. It’s a wonderful method for creating art. However, human hands are the true engine for creating art. They can create sculptures, paintings, films, etc.

   Vectors are necessary — and must be part of the styling engine, not an additional add-on. SVG has effectively been abandoned by all major browser makers. Both Flex (mx:graphics) and WPF (xaml) offer this functionality as part of their styling engines, and it’s clearly essential.

   And saying that AIR is just flash with a new dress is simply wrong. I won’t even go past that.

3. 

   Oliver

   February 1^st | #

   SVG has been effectively bandoned by all major browsers if by all major browsers you mean IE alone. Firefox, Safari, and Opera all support at least some of SVG, all are working hard on supporting all of SVG. Saying it’s been abandoned is as wrong as you believe my statement about AIR is.

   Vectors are content they are not style, by definition.

   And i’m not sure what you believe AIR adds that Flash doesn’t already have, other than perhaps the ability live as a “standalone” app — maybe i’m wrong, but certainly none of the demos i have seen have made me go “ooh, wow, that’s new”

4. 

   Oliver

   February 1^st | #

   Oh, and re: webkit transforms: you can scale, move, animate style properties. No you cannot animate arbitrary values though — but then these are css animations so it only makes sense for it to animate style. SVG supports all of your desired animation, although i think Opera is the only browser to support it.

   Of course these things will change over time — with the exception of IE, the major browsers compete on features and performance so are continually striving forward. However unlike competition between say Silverlight and Air the goal for a browser is to gain users through always being better than the competition — given these technologies are all standardised vendor locking is not, and cannot be a goal.

5. 

   Oliver

   February 1^st | #

   Anyhoo, you have probably guessed that i believe the future is in the open standards rather than proprietary technology, even if proprietary tech is ahead at the moment (eg. i know that at the moment Air and Silverlight can both do a more than html, css, etc)

6. 

   Oliver

   February 1^st | #

   (And apologies for the first post — on rereading i realise it is troll like at the end :( )

7. 

   David Malouf

   February 1^st | #

   Hi Kyle,
   great piece. I think you would find that there is a community of people adjacent to the UI Developer world thinking in greater detail and with more direct contact to the practice of the design side of UI and Interaction. The Interaction Design Association has a global community which speak a lot on the topics this piece is about. I really recommend it to you.

   http://ixda.org/

   Great article, BTW and I agree with almost all of it. I think that your insights can be really heavily supported within the confines of interaction design and then applied through UI Development.

   – dave

8. 

   Jeff Seager

   February 1^st | #

   Great insights, yet I wonder where you see accessibility fitting with all this. Flash and its progeny require accessibility accommodations that many developers simply overlook, and I don’t see this changing.

   It may or may not interest you to know that 6.5 million Americans over age 65 now have age-related vision loss. In the next two decades, rates of vision loss from diseases like age-related macular degeneration are expected to double as the nation’s 78 million baby boomers reach retirement age. Many of us will be accustomed to interacting professionally and socially on the Web.

   It remains to be seen whether daily interaction with computers will hasten the onset or prevalence of such disabilities. I think it may. But even if it doesn’t, what will become of us if these technologies become so ubiquitous (as I think they will, if they haven’t already) that the majority of us rely on them for everyday functions of our lives? The person affected may be you or me, my wife, my child or friend. So this should matter to all of us. None of us is immune or immortal.

   I think accessibility must be considered hand-in-hand with each advancement, and I’d be interested to know your own thoughts on that, Kyle.

9. 

   Tzek

   February 1^st | #

   Interesante. Aunque creo que apostar por tecnologías mencionadas en explícito puede ser riesgoso por lo cambiante que es el futuro… pero las ideas generales y conceptuales, muy posibles.

   A ver que sucede.
   Saludos desde México.

10. 

    Kyle

    February 1^st | #

    Oliver: I think you’re a lot confused on a lot of points, and my guess is that you haven’t really built an application with Flash, WPF, or Cocoa. Vectors are certainly style (unless you’re going to argue that background images should be put in the HTML (content) now. AIR is far more than a fancy wrapper (hell, we’ve had standalone executable .fla’s for half a decade now). I urge you to branch out and try some of these new technologies before setting fire to their existence. I think you’ll find much of your viewpoints change.

    Jeff Seager: I believe most people use the word accessibility as an excuse not to learn new languages in all honesty. Flash can be accessible in the same way HTML can be accessible. Neither are accessible in the fashion that they are truly useful to disabled persons. The idea of providing your content in an easily consumed format allows people to build custom applications targeted at the disabled, and all your problems are solved.

11. 

    Geof Harries

    February 1^st | #

    While the Flex applications I’ve been building this year have yet to be publicly deployed (for various reasons) I can say the work experience has completely changed my perspective on interaction design and what end-users can and should be able to do with an application.

    Using Flex means I don’t have to be tied to a web browser, HTML, Javascript and CSS or limited by their technical restrictions. I’m all for constraints, but being allowed to solve a problem in a new way, unbound by rules and regulations set out nearly a decade ago - here’s looking at you, W3C - is refreshing and heck, just a lot of fun. You need that change after a while or you’ll grow incredibly stale.

    I always keep people (and thus principles of accessibility, usefulness and usability) in mind when I’m designing and developing in Flex. These are lessons learned from years of wrangling HTML/CSS. There is no rationale reason why you shouldn’t be able to transfer these values across to, dare I say, the new paradigm.

12. 

    Anonymous

    February 2^nd | #

    Kyle, I have to say you are right on. I think you’ve made a very good case in support of all risks that are posed in building either in CSS or Air or Flash. I think we are still in the infancy of technology built for online, pda’s, or anything interactive for that matter. We have so many variables that can shift an idea 1 degree or 180 degrees.

    As long as technology continues to change we will need to change with them.

13. 

    Shawn Medero

    February 4^th | #

    WebKit’s CSS animation and transform extensions handle things like:

    • rotate
    • skew
    • scale
    • opacity
    • ease-in/out

    Source:
    http://webkit.org/blog/138/css-animation/
    http://webkit.org/blog/130/css-transforms/

    (And I’ve seen them in action with my own eyes. More impressive is seeing them work in-combination with HTML 5’s video element… also supported in WebKit nightly builds.)

    The WebKit team appears to be actively documenting these extensions into a specification anyone can use. A demo of the extensions to the HTML Working Group received very positive remarks from Internet Explorer, Mozilla, and Opera folks… so maybe there’s an opportunity we will see cross-browser implementations.

    I just wanted to clear up any misinformation on those… I’m not saying don’t use Flash or Flash isn’t the future… nothing like that at all. Just a friendly heads up.

14. 

    links for 2008-02-05

    February 5^th | #

    […] Manifesto of the UI - Warpspire Over the past few years (we’re talking less than 5), we’ve seen user interfaces across the digital world morph from a static experience into highly dynamic interactive experiences. Web sites like Facebook and MySpace have proven that interactivity and (tags: ui ux design interface accessibility) […]

15. 

    Kyle

    February 5^th | #

    Shawn: I appreciate your enthusiasm for WebKit, and I do enjoy that so much effort has been put into the WebKit engine as far as lightweight animations. But please, they are just that: lightweight animations.

    It does not come close to handling the raw power necessary of a fully functional modern animation engine. Think about trying to create things like 3D carousels, genie effects, morphing vector masks, etc. I know people love to tout that their own tech can “do it too!!!!!”, but in this case we’re just talking about completely different magnitudes of scale :)

    Most importantly, the webkit animations currently are one-off’s. There’s really no dynamic controlling of them (how would you animate something to music or user input?). There are structures for pre-defined looping, etc — but it’s just one-off’s that they’ve provided for. Again, nothing against the WebKit team — we’re just talking about apples and oranges. WebKit CSS animations are for lightweight animations, not a fully functional animation framework (think Core Animation).

16. 

    Jason Marsh - Website designer

    March 19^th | #

    There is certainly a lot to say about semantics and the way we interact with information.

Name

Email (will not be published)

Website (optional)