Friday, 30 January 2009

How Sun could fix Swing and promote innovation and unification in the UI space

I saw Jonathan's post on Swing 2.0 and it was late, I'd had a nice glass of wine so I fired off a wacky idea on twitter which got some interesting feedback. I'd like to just expand on the background of the idea some more as I'm not so sure it is so wacky an idea afterall. Its feeling more and more like a no brainer and complete win win.

I'm afraid this post is gonna be quite long. (Holy crap - no blogging for ages and then two big posts in a week!) - first some background...

Programming Rich Java Applications Is Sucky
Imagine the scene. You're part of a team of Java developers; you're tasked to write a rich application. It might be a commerical app or some intranet enterprisey thing, or some rich application to improve the user experience of some web app (like the iphone versions of facebook/twitter/gmail etc) - whatever. So what do you do?

You've a mindnumbingly broad choice to choose from - here's just a few options...
  • Swing
  • SWT/JFace/EclipseRCP
  • web app with some combination of Ajax/JavaScript/GWT libraries
  • JavaFX?
  • Flex/Flash
Each have their own strengths and weakesss. Though they all totally different. Write for one of those then later on you're requirements change (making one of the other options more attractive) and you've gotta scrap it all and start over.

Write once and run everywhere - except if your application has any kind of user interface.

This is maybe why so many Java folks like writing server side apps; as the client side in the Java ecosystem is such a mess.

Helping Swing Developers
Lets imagine you pick Swing to develop rich applications. In its day, Swing was a good piece of work by Sun. However that day was in the nineties :)

I've done a fair bit of Swing in my time and worked on Swing projects; in my experience you can spend years trying to tart up a swing app and it still looks - well like Swing. You can get a day of a good web designers time and make any web application look stunning via HTML/CSS/images. It just seems so much easier to hack up HTML/CSS to look great. Just compare some great web 2.0 applications out there to your typical Swing UI. As an end user which would you rather use?

As a developer one of the big PITAs with Swing is layouts, styling and working with images and links. The standard layout managers suck. (GridBagLayout anyone?). Yes I know there's a zillion layout manager implementations out there - all doing different things well with different results & rendering bugs - and which one do you choose?

Generalizing hugely but most of the work that goes into most data entry-ish rich applications is creating loads of screens which are full of text, images, links and some rich widgets with plenty of documentation, popup help, cheat sheets and then laying it all out and styling it nicely. Swing is lousy at these UI basics. Sure its got a good JTable and JTree, something that many a web developer will probably have been envious of at some point or other when hitting gremlins in their JavaScript table/tree - but at laying out forms, dialogs, help screens, documentation and working with links and images - well its kinda sucky.

Swing developers should be using HTML/CSS for this - its great at it. The web browser is the best layout manager we have - and HTML/CSS is the best standard for laying out widgets. With HTML/CSS there's a ton of tooling out there too for viewing/editing/styling. You can then pass the HTML/CSS to your web designer while the Java Swing hacker focusses on the business logic or the complex widgets.

Plus any time you spend hacking HTML/CSS - being a simple, standard text format used inside web browsers means you can reuse your work in web applications as well as rich applications. You don't spend your time hacking up some Swing Java code thats only useful inside a Swing application.

My Proposal
So my proposal is, in Java 7 we integrate WebKit into the JVM. Why WebKit? Well its smaller and easier to integrate than Gecko; Apple chose it for Safari & iTunes and the iPhone and Google chose it for Chrome and Android. If its good enough for a cell phone surely we can fit it into a JDK/JRE for a rich desktop application?

If Sun wanted to get clever, the browser engine could be pluggable; letting folks pick different versions of WebKit or FireFox/Opera/IE - but lets not go there right now - lets just assume a recent WebKit version is baked into the JRE/JDK.

Some would argue it should be optional; but then given the size of the JDK/JRE there's a ton of not-that-useful stuff in there; I'd rather include it by default over loads of whats there :) But then the JDK/JRE is getting more modular right - so lets leave the optional nature of WebKit for when the JDK/JRE is truly modular - right now its huge; so if a cell phone can include it, so can a JDK/JRE :)

Next we have a standard Java API for creating a BrowserPanel which can be hosted inside any AWT/Swing/SWT/JFace based UI. So a browser window becomes a standard embeddable widget in Swing and SWT/JFace/Eclipse. There should also be an API for controlling the browser (loading/saving URLs and interacting with the DOM).

Finally I'd like any AWT/Swing/SWT/JFace/JavaFX component to be usable within the DOM of WebKit. So I could create a nice HTML page with nicely formatted and CSS'd text, images, tables but then include, say, a JTable and JTree in the middle somewhere; letting the browser do the layout stuff.

This last requirement might seem complex but thats exactly what Flash and JavaFX are already; plugins which create custom UI widgets within the DOM.

For those that are aware of GWT Hosted Mode; this proposal is very similar - embedding a web brower inside the JVM as a single process. The main differences are
  • we need a standard API to spin up browser windows and embed them inside any Panel and to be able to bind any UI Panel inside the DOM easily (e.g. finding a div by ID and binding it to a JTable)
  • we don't need the restriction of being able to turn the Java bytecode into JavaScript - as for standalone Swing/rich apps you can can leave it as it is :). So any Java code could work with the browser. So none of the GWT restrictions would apply - plus you could use Groovy/Ruby/Python/JavaScript/whatever
Benefits for Swing Developers
Swing would instantly get an awesome, standards based layout engine capable of easily rendering styled text, images and widgets while dealing with other kinds of media. Through the use of CSS Swing applications would be easily styled using all the existing HTML/CSS tooling.

Swing developers would then have the best of both worlds; their Swing widgets still working when they want to use them - but having the great power of the browser window around the widgets to make the layout easy, powerful and great looking.

Since the JVM would now have a standards based browser; and it can already run JavaScript (and pretty much all other languages) easily - any Swing application could make use of any of the current GWT or JavaScript frameworks out there. There's a ton of innovation happening in the JavaScript/GWT space right now leading to ever better UI capabilities. e.g. take a look at SmartGWT and jQuery to name but 2.

Benefits for Web Developers
So how will this move - aimed at helping Swing developers help Web developers? The common PITA for web folks is often the browser imposes restrictions (for good reason) such as
  • only being able to interact nicely with the single domain that serves the web page
  • not able to interact with the file system
  • only having 2 threads for communication
  • limited connectivity (kinda hard to use native sockets, Java web stacks, JMS, native JDBC etc)
  • unable to use things like Java2D/3D without via plugins
Plus there's dealing with the large number of different browsers of different capabilities & configurations & versions to support.

Now if you're making a public application; you're probably gonna have to deal with this (though now we're seeing custom apps for the iphone and android - maybe rich applications for public web sites are gonna be more common?). However loads of folks build web applications when they are really building internal, intranet applications.

So why not turn that web application into a desktop application making it run in a JVM? You can then package up your application with the exact, known browser you use and test with - and have access to all the features of the Java platform in your UI; whether its Java2D/3D, Swing/JFace, JavaFX or JMS/JDBC/JAX-WS etc

How this application is then made available to your users could come in different flavours; an application they install, Java Web Start (which might become cool again) or maybe even via JavaFX inside a browser?

So this could give a big safety net to folks building web applications. If you hit a browser limitation such as (say) a Canvas limitation? Just provide a Java2D version and ship a desktop application or Java WebStart it? It could end up that the JVM turns out to be the best place to host a web application reusing HotSpot?

Benefits for UI Developers
This single act of unifying the browser with Swing (and SWT/JFace) would provide a massive unification of standards based technologies for most UI development. More of the time spent building UIs would lead to creating more reusable standard artefacts (e.g. HTML and CSS) which are reusable in other contexts (e.g. transform HTML into PDFs or reuse the embedded microformats etc).

In these days of Dependency Injection; we're used to having a single code base then switching component implementations in different circumstances and environments; so we can start building more UIs which can be both web application and rich UI with each UI making the best use of its platform (zero install for web app, no limitations for desktop app).

Previously changing an application from a web application to a desktop application or vice versa was a massive undertaking (near total rewrite). Now it'll be pretty trivial to switch from one to the other as your requirements change. Applications will no longer be locked into one UI technology unable to move. UI code will become more aligned with the Write Once Run Anywhere mantra.

If folks create rich applications and web applications - there could be massive code reuse. While a web application and a Swing/JFace application are clearly going to have differences; they may use different views, components and features - there could be a huge amout of reuse of code. Even if the code reuse is only 30-50% its is big a saving; and we all need to do more with less these days. Though I can see - as UI frameworks improve, more and more chance of better code reuse across both.

Benefits for Sun
Folks have often said things in the past about Sun like
  • Sun doesn't get the desktop, its not its core business
  • Sun can't compete with Microsoft in its back yard (the desktop)
Sun is tightening its belt as is prudent for any company to do in the current climate. Sun has already de-funded SwingX. Also so far all the feedback and responses I hear from non Sun folks about JavaFX is at best lukewarm.

This move however would be a massive, positive move in the UI space and earn Sun a lot of kudos IMHO. Sun could then leverage all the innovation thats taking place in the browsers and WebKit in particular (HTML5, SVG, microformats et al) along with in the JavaScript worlds.

It would also mean there's not that much for Sun to do (other that integrate WebKit and provide APIs to control the browser and expose the DOM). All the innovation can then happen above this layer by the Swing, JFace & web developers and the WebKit & JavaScript communities. Immediately the Java ecosystem could then reuse all of the amazing work being done by the webby folks in JavaScript or by the GWT ecosystem.

i.e. the Java platform would then become a cool place to develop applications for (removing all the limiations with web browsers) with loads of buzz -and Sun doesn't really have to do much other than take all the credit :)

Heck they could even brand the WebKit-in-JVM as part of the JavaFX brand; so JavaFX doesn't have to beat Flash/Flex & Silverlight in the market to avoid looking like a turkey. JavaFX Platform (or whatever we call it - basically Java7 + WebKit) could turn into an instant success (anyone doing Swing or JFace would probably start using the embedded browser for better layouts & rendering) and if that helped the JavaFX Plugin (the JavaFX-as-browser-plugin) make some inroads to Flash/Silverlight its an added bonus. (Whereas right now JavaFX as just a browser plugin looks setup for failure to be honest, I don't see it ever eclipsing Flash).

Note its worth mentioning that there's similar moves to DOMs and whatnot being experimented with in the Eclipse ecosystem; who knows we could maybe finally unify on a single DOM?

Benefits for the Java Platform
Being able to build rich desktop and web applications in any language - without any browser limitations would be a massive boost to the Java platform; allowing folks to reuse all the good stuff from the OpenWeb and the Java platform (language diversity, IDEs and tooling, connectivity, Java2D/3D and even JavaFX).

We'd also hopefully see significant cross-pollination of the Swing / JFace folks and web folks to help UI innovation (which Sun doesn't have to try do itself).

If nothing else hopefully it will encourage more folks to write web applications and desktop applications to take advantage of both environments to provide better user interfaces for our users and customers for less effort.

Sounds like a win win for everyone doesn't it?

Update - JWebPane
So I should probably pay more attention to the JavaFX stuff :)

It seems that there is such a thing called JWebPane (see page 27 onwards of this presentation) which has been announced - so maybe this is gonna actually happen soon! :).

IMHO JWebPane sounds way more important to the Java ecosystem than JavaFX - I really hope it gets enough backing and funding from Sun to get completed and released real soon! Its not good to see nothing at all being released to the community despite it being announced about 7-8 months ago - so fingers crossed that whatever the legal/licensing issues are holding it back get resolved really soon.

We've just gotta lobby Sun to release it ASAP - then include it in Java 7... :-)

20 comments:

Boris Bokowski said...

Hi James,

Yes, what you describe is indeed very interesting. And the technology is available today!

SWT has had a Browser control for quite some time, and with a bit of extra install overhead, you can get an embedded Firefox on all supported platforms. Also, as of 3.5 M4, SWT has pretty good API for JavaScript<->Java communication, see the New&Noteworthy at: http://download.eclipse.org/eclipse/downloads/drops/S-3.5M4-200812111908/eclipse-news-M4.html

We have explored the area of how best to embed existing web UIs in an application like Eclipse, with great success. I will be talking about this at EclipseCon in March: http://www.eclipsecon.org/2009/sessions?id=636

James Strachan said...

Thanks for the heads up Boris! Yeah I was aware SWT/JFace can host a browser - and it can host native COM components and the like which is great.

My post was really about what should be in the JVM as standard; that Swing developers can use (or SWT for that matter) without having to add platform specific libraries to your PATH to work etc.

I'll look out for the slides of your talk though; I'm hoping the meme of using WebKit/Gecko as the main layout/rendering component in rich UIs on the Java platform can spread - and maybe even share the same common API to interact with the browser

blackdog said...

I agree wholeheartedly with your thoughts and particularly this being more important than JavaFX. Instantly, Sun will have thousands more developers developing for the JVM, instead of trying to convince them that somehow JavaFX is better. The fact is, when you add jQuery (or alternative) over the DOM it's really quite pleasant to work with and if Sun could team up with this initiative for a common JS server side API (http://groups.google.com/group/serverjs) - really would be a nice way to do desktop apps apps.

At least the work is in progress, this was the last info I saw on it, http://weblogs.java.net/blog/alex2d/archive/2008/12/jwebpane_projec.html

bd

James Strachan said...

blackdog: totally agreed!

Boris: have you taken a look at the Java API for interacting with WebKit in Android by any chance? I'm wondering if we can try get the Java-browser APIs in some kind of sync so we could maybe try share a common API to interacting with the browser, the DOM and JS?

e.g. have a look at the comment by pgmglv in the comments here...

http://weblogs.java.net/blog/ixmal/archive/2008/06/using_jwebpane_1.html

I guess the API of JWebPane is totally mute until something is actually released publicly :)

Jessta said...

" The web browser is the best layout manager we have - and HTML/CSS is the best standard for laying out widgets."
ummm...HTML/CSS is terrible for UI. It's a bunch of nasty hacks to get a document layout format to work for proper user interfaces.
This web app stuff is getting out of hand.

James Strachan said...

Jessta I'm keen to know which layout manager you've been using for Swing that can layout widgets much better than HTML and CSS

Mario Grgic said...

Sun has you beat. Java FX SDK is rumored to have JWebPane which is supposed to make it into the JDK as well (though most likely not 1.7) that is meant to replace JEditorPane (which can only render HTML 3.2) and is based on WebKit.

James Strachan said...

Thanks Mario! Yeah I spotted JWebPane after I posted the first version of this post - I updated the last section on JWebPane.

Though note still its not been released in any alpha/beta form despite being announced 7-8 months ago. I've got my fingers crossed Sun deliver it!

Oliver said...

Apple didn't "choose" WebKit for Safari -- Apple wrote it, starting from KHTML as a base (5 or 6 years ago), more or less all of webkit is written *by* Apple nowadays.

James Strachan said...

Thanks for the correction Oliver; I guess I meant Apple didn't choose Gecko :)

Jacek Furmankiewicz said...

HTML/CSS for layout? That's insane.

MigLayout beats HTML/CSS for layout management by a factor of 1000.

Axel Rauschmayer said...

The idea is great in principle, but HTML is not there... yet! Layout with HTML is incredibly painful (I love how JGoodies and MigLayout do it), it is practically impossible to do well for complex applications [1]. But that will change before long and then Java needs to have a plan. Browsers are going to become a really cool programming environment in the future (see Palm Pre, Google Gears, client-side SQL, etc.). And being able to use Java as the J in Ajax would be cool, because then one could switch between Desktop app and web app easily. Or just always use a web (or web-like) environment for deployment.

[1] 2ality.blogspot.com/2008/12/future-of-ajax.html

Developer Dude said...

I have to disagree.

At least with Swing I know that what 99.9% of what I write will work on the three main platforms I develop for (Windows, Linux, OSX) and it will work the same.

I cannot come close to saying the same for any web client regardless of what I use (GWT, whatever). GWT is a big improvement, but mostly in the fact that much of the time I can work in Java in hosted mode - not that HTML/JS/CSS is better than Swing - it isn't.

I have developed in both Swing and for web clients (using GWT, or JSP, or Tapestry) and I am several times more productive and much more robust when I use Swing than any web client framework.

That isn't to say that Swing (or SWT for that matter) is the answer for all apps or clients, it isn't, just that when it is the answer I much prefer it over anything else. Sure it can use improvement and cleanup, but I find it much more preferable than the alternatives and there are many third party add-ons (layout managers for example) that greatly improve it.

James Strachan said...

Developer Dude

Most of the pain of web applications IMHO is dealing with a zillion different old browsers and bugs. However if the JDK came with a recent WebKit - which works the same on Windows, OS X and Linux then it'll be just as cross platform as working with Swing right?

BTW I'm not advocating we use web frameworks instead of Swing; I'm advocating we build a neat API - at least as good as Swing and hopefully better - which underneath can reuse the *good* stuff from WebKit and get a best of both worlds. Essentially combining the best bits of Swing and WebKit so users have the simple, productive programming model while being able to use HTML markup, CSS and standards when and where applicable etc.

If folks have specific use cases for something like MigLayout - then hey use it. I'm not advocating scrapping Swing - more enhancing it to allow WebKit / HTML / CSS / DOM / JS to be used as and when developers choose to.

Dominique D. said...

Your proposition sounds like very close to personal ideas and a recent post of mine : Swing browsers - other interesting DSL Swing projects to follow (and related indirectly also to Proposing a Swing DSL from JavaFX ?).

What I call "Swing browser" is somewhat: UI declarative programming + Swing components + XHTML-or-specific DSL for composition, all the stuff being more or less similar to a web browser, but with Swing underneath.

The more polished and *running/living* example/implementation of such Swing browser is Sage browser from Don DeCoteau (see my post for other examples, but not as polished/finished as Sage browser unfortunately).

I have also written in 2006 about Revisiting XHTML as a base (?) for XUL-like programming: "XHTML may simplify Java client development as XHTML programming model is common (similar to HTML) and because XHTML Java support and use are growing". This being said, while Java could leverage XHTML for gluing/composing components, I think XHTML might be adapted if, for example, some want to continue to use Swing layout managers, but on XHTML components.

Well, I hope JWebPane is going to make it appear at last, because I am also interested into using quite similar technologies for web and desktop development.

Boris Bokowski said...

FYI, here is a recent announcement of an Java-embeddable browser component based on Google Chrome:

http://www.jroller.com/myeclipseblog/entry/chromius

Maurits said...

Maybe we should include not only webkit but also apache. I think a cross-platform VM that lets you package a web app as desktop app might win the battle..

James Strachan said...

@Maurits there's plenty of open source pure Java web server & servlet engines available, so its trivial to embed a full web app inside any Java application

Fahad said...

Hi,

I am ready to pay $1,000 USD for this domain dom4j.org please renew it for me. I will pay you advance.. This is not a spam email please help me i need this domain.

I can pay money advance.

Thanks
Fahad

Fahad said...

Please Help today is a last day to renew this domain help me out i am ready to pay.

Thanks