A favorite feature of mine in the next Apache Camel is a new feature to use file name patterns in the file component. What we have introduced is a new language, the file language, so you can leverage this feature anywhere in Camel where it uses Expressions, Languages etc. The file language currently support both the file and the FTP component as well.
As a lot of integrations is file based you need to be able to read (consume) or write (produce) files. Then you need to be able to specify file names. What's new in Camel 1.5.0 is that you can express this using patterns, for instance directly in the configuration of your endpoints. Prior you had to do this in Java code.
Simple example
Lets image you need to move consumed files into a backup folder after processing.
Now you can express this using the pattern: backup/${file:name}
Well then you need to use .bak as extension, and the pattern is:backup/${file:name}.bak
Okay next issue is that the backup folder should be grouped by dates using the yyyyMMdd pattern. Well the expression support thejava.text.SimpleDateFormat patterns also.
The pattern is:backup/${date:now:yyyyMMdd}/${file:name}.bak wherenow means current date. You can substitute this with different commands such as file for using the timestamp on the file instead.
Advanced example
In this sample you need to produce files (save) and you need to save the file using a generate unique filename. We use the divide and conquer pattern for this so you create a POJO class that generates the unique filename, then you as the developer have the full power how to do this. And then its very easy to unit test as its plain POJO that can be tested very easily with JUnit. Next step is to express this as a file pattern. As your POJO is a bean we use the bean langauge to invoke your pojo.
So the pattern is: myfile-${bean:myguidgenerator.generateid}.txt
Where myguidgenerator is the bean id of your POJO and generateidis the method name. You can omit the method name if there is no ambiguity which method Camel should invoke.
Camel routes
This feature is configurable directly in camel routing on your endpoints. So what we can do now is:
from("file://inbox?expression=backup/${date:now:yyyyMMdd}/${file:name}.bak").to("bean:processFile");
The route will consume files and process the files in the processFile bean, that is a plain POJO class. After processing the files is moved (renamed) using the pattern in the expression. So the file is moved to the backup subfolder, for example: backup/20081014/report-october-2008.bak
More to come
Well there is a ton of new features and improvements in Camel 1.5.0. Check out the current release note.
Tuesday 14 October 2008
Liking the File language in Camel 1.5.0
Thursday 9 October 2008
Using Guice as a JNDI provider
Tuesday 7 October 2008
Adding support for @PostConstruct, @PreDestroy and @Resource to Guice
- @Resource from JSR 250/EJB3
- @PersistenceContext from JPA
- the various JAX-RS annotations like @Context, @PathParam etc
- WebBeans @In
- Spring's @Autowire
- Apache Camel's injection annotations like @EndpointInject or @Produce
Enterprise Integration Designer preview
Thanks to Enterprise Integration Patterns, architects have a clear and succinct way to describe integration within complex systems. It certainly beats a lot of MS-Word documents and monster integration architecture diagrams. And, with Apache Camel, we’ve got an open source runtime that allows us to create routes by chaining EIPs together, using Spring XML or Java fluent builders. That’s fantastic, but EIPs are as much about sharing pictures as creating message routing graphs, and where are the pictures?
We’ve made some tools that will allow you to load up Camel Spring XML files into Eclipse for visual inspection and editing. You can also create EIP diagrams in your Workbench and save them as Camel configurations. Finally, we’ve put in some debugging capabilities to allow you trace through the paths a message will take through your EIP graph.
We are earnestly seeking feedback - check out the FUSE Integration Designer Preview page to download the Eclipse plugins, or check out the video links under the Training Videos at the bottom of the page.
Monday 29 September 2008
Using Camel with ServiceMix Kernel
Thursday 18 September 2008
Apache ServiceMix Kernel 1.0.0 released!
Apache ServiceMix Kernel 1.0.0 has just been released.
Apache ServiceMix Kernel is a small OSGi based runtime which provides
a lightweight container onto which various bundles can be deployed.
Amongst the list of supported features, Apache ServiceMix Kernel supports:
- hot deployment of OSGi bundles, exploded bundles or custom artifacts (spring xml configuration files support is provided)
- services configuration stored as property files are monitored and provided as standard OSGi configurations
- a centralized logging back end supported by Log4J, ServiceMix Kernel supports a number of different APIs (JDK 1.4, JCL, SLF4J, Avalon, Tomcat, OSGi)
- provisioning of libraries or applications can be done using simple commands via simple xml descriptors
- native OS integration as a service so that the lifecycle will be bound to your Operating System.
- an extensible shell console to manage services, applications and libraries
- operations on the console can be done remotely via a secured and encrypted channel
- a security framework based on JAAS
- new instances can be created using a single command line
This release, with the detailed release notes, is available at:
http://servicemix.apache.org/kernel/servicemix-kernel-100.html.
I'm particularly pleased to get the 1.0.0 release out; it makes it really easy to have hot-redeployable Camel routing rules - just edit your Spring XML when using the expanded bundle mode and ServiceMix will auto-redeploy your routing rules dynamically!
Tuesday 9 September 2008
Whats new in ServiceMix 4 and Camel
Monday 1 September 2008
better stack traces in Java with log4j - including the jar file and version number!
Given that its possible to figure out the actual version being used of each class - either using the package information from the MANIFEST or by finding the jar the class came from, it seems logical to include the version information into a stack trace at the end of the line.
One day the JVM might actually do this for us :)
In the meantime, I've created a patch for log4j to add this feature; it appends an optional String to stack traces printed via log4j which includes the jar file name if it can be deduced plus the Java Package version the line of code comes from.
Here's an example...
org.apache.log4j.config.PropertySetterException: HelloSo you can see what version of junit is being used (despite there being no manifest information) along with the JDK implementation version.
at org.apache.log4j.spi.ThrowableInformationTest.testStackTracePackageName(ThrowableInformationTest.java:306)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [1.5.0]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [1.5.0]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [1.5.0]
at java.lang.reflect.Method.invoke(Method.java:585) [1.5.0]
at junit.framework.TestCase.runTest(TestCase.java:154) [junit-3.8.1.jar]
at junit.framework.TestCase.runBare(TestCase.java:127) [junit-3.8.1.jar]
at junit.framework.TestResult$1.protect(TestResult.java:106) [junit-3.8.1.jar]
at junit.framework.TestResult.runProtected(TestResult.java:124) [junit-3.8.1.jar]
at junit.framework.TestResult.run(TestResult.java:109) [junit-3.8.1.jar]
at junit.framework.TestCase.run(TestCase.java:118) [junit-3.8.1.jar]
at junit.textui.TestRunner.doRun(TestRunner.java:116) [junit-3.8.1.jar]
at com.intellij.rt.execution.junit.IdeaTestRunner.doRun(IdeaTestRunner.java:94) [idea_rt.jar]
at junit.textui.TestRunner.doRun(TestRunner.java:109) [junit-3.8.1.jar]
at com.intellij.rt.execution.junit.IdeaTestRunner.startRunnerWithArgs(IdeaTestRunner.java:22) [idea_rt.jar]
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:118) [idea_rt.jar]
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:40) [idea_rt.jar]
e.g.
at java.lang.reflect.Method.invoke(Method.java:585) [1.5.0]means we found the package number but couldn't find the jar file but we found the implementation version of java.lang.reflect to be 1.5.0. Whereas
at junit.framework.TestCase.run(TestCase.java:118) [junit-3.8.1.jar]means we found the jar file but the jar file has no manifest version information.
Am hoping one day this gets into some log4j release! You can find the patch here if you're interested.
Friday 29 August 2008
am loving Beyond REST and the PIMP protocol
My suggestion is to think of this instead as another form of caching. All we really want is a header that tells the server that we are interested when a particular resource has been updated and how to tell us. The server can then either understand that header and acknowledge in the response that it will notify me. Here is my strawman:Request:
...
X-Cache-Callback:http://www.javarants.com/notify/joshua.schachter.org/atom.xml;SECRET
Response:
...
X-Cache-Callback: OK
Then if that resource is updated the service is expected to either HEAD the callback as a notification or POST the new contents of the resource, servers choice. You could later add semantics for merely updating the resource vs replacing it wholesale. I would also think about adding the ability for the server to specify a timeout after which you are free to poll again if you haven't heard anything on the assumption that sometimes the service may lose the state associated with your subscription.
Am thinking rather than returning OK the server returns the amount of time before the client has to re-issue the subscription to keep it alive. So the server can decide the maximum subscription time. Good PIMP servers (PIMPS :) might wanna make this quite long to reduce the polling overhead.
I also love the simplicity of the HEAD or POST to differentiate a notification of change to a notification-with-the-data.
I've long wanted a 'SUBSCRIBE' verb in HTTP for doing this kinda thing; but I think your cache-header approach is cleaner - as folks can either keep polling and/or subscribe for the update notification.
The nice thing too is that it allows easy migration to PIMP without introducing any overhead or new traffic; that clients continue to poll as normal - but they advertise themselves as being PIMP aware. Then eventually when one day the server becomes PIMP aware the clients receive their notifications (and then hopefully they scale back their polling :) - otherwise they can stick to polling.
Also webmasters can monitor their traffic looking for PIMP headers to know when it'll make sense to upgrade to PIMP. Not everyone is gonna need PIMP and it'll be a no brainer from looking at your logs to determine both when you've sufficient mass of PIMP enabled pollers along with knowing what the reduction in polling traffic upgrading to PIMP would save you.
I'm with Sam in the thinking of this as another form of caching. In implementing PIMP some folks might be able to create update notifications internally in their system when resources change to push out change messages into some kinda queue for posting to the callback URL. This would involve significant work for many sites though.
However it'll surely be pretty trivial to just install a PIMP-enabled caching web proxy inside your data centre in front of your servers - that does the usual cache thing, but also detects these extra PIMP cache headers and does a background poll of resources (respecting your existing cache & time to live headers) to detect changes both to update the cluster of front web caches (so non-PIMP pollers get more real time data) but also to drive the pushing of updates out to PIMP subscribers.
i.e. I can see this as a pretty easy upgrade to most web sites - folks just update their front end web proxies to a PIMP-enabled version and hey presto you can now support PIMP consumers. Am sure the web proxies could include an XMPP firehose too pretty easily for heavy hitters.
It should be pretty easy to hack the web proxies to do this I'd have thought? Even the problem thats been noted earlier in this thread - of trying to push updates to a URL endpoint might be slow, unresponsive or unavailable - the web proxies have to deal with already right in case a *local* server is borked.
Anyway - its a very interesting blog post and particularly the comments. Interesting stuff.
Thursday 28 August 2008
Running Nexus with launchd on OS X
This gave me the excuse to finally take a quick look at using launchd which is the preferred OS X way to run daemons and services. First look it appears cool and much nicer than rc/init.d/xinit.d et al.
Here's a really simple always-run nexus script, sonatype.nexus.plist. Just drop it into your ~/Library/LaunchAgents/ directory and you should be good to go.
From the command line you can then do this if you want to boot it up straight away - but it should restart on reboot (which I'll test out next time I've gotta reboot :)
launchctl start sonatype.nexusI did try get clever and wait for port 8081 to be used before booting up nexus but couldn't get it to work properly so figured just starting it up on boot was easier :)
Wednesday 27 August 2008
Neat eclipse templates for working with the Java or Spring DSLs for Camel
Another great Camel tutorial from Claus
Thursday 24 July 2008
Apache Camel 1.4.0 released with 261 new features, improvements and fixes
Just after 3½ months since release 1.3.0 of Apache Camel we have been very busy and have resolved 261 tickets.There's tons of stuff in there. One of the most trivial changes to implement but one of my 'DOH why didn't we do that a long time ago' changes is the new tracer; very handy! Great work camel riders!
Check out the release notes and go grab it from the download page.
If you are not familiar with Apache Camel then check out the FAQ or the Getting Started guide.
If you have more time and a fresh brew of coffee then I encourage you to read a somewhat different tutorial that focus on introducing Camel into an existing project step by step. The tutorial is work in progress, so check it out from time to time. The tutorial is based on a real life use-case.
If you have less time check out the regular tutorial that demonstrates how Camel easily work well together with Spring for message exchanges over JMS.
If you have questions about Camel then please check out the forums/mailinglists.
And then the answer to the mother of all questions about Camel.
Monday 23 June 2008
Are people blogging less?
Tuesday 17 June 2008
Setting your bash command prompt to subversion or git repo info
Tuesday 3 June 2008
TimeCapsule & TimeMachine saved my ass!
Luckily the TimeCapsule had all my stuff so I could use the Migration Assistant to restore my apps & user accounts from the backup on the Time Machine. The UI did freak me out a bit - when the thing started it kept saying 'connecting...' then hanging for ages. I kept thinking it wasn't working so trying all kinds of things. Turns out, its just wacky slow. The next screen shows the list of user accounts - again thats wacky slow - takes maybe an hour for a 100Gb backup to show the few user accounts in the backup!
The 2 massive delays in the UI are kinda irritating; I mean why can't it keep a little index of what machines & users its got in the backup and let me pick 'em in a second then leave the thing for a few hours while it restores?
Anyway - all is well - once I got past the 2nd screen, chose the things to restore I left it chugging away for a few hours and I'm now restored. The only thing I've missed so far is I had to reinstall my VPN client as it shoved stuff in /System and maven is hardwired in /usr/bin to point to mvn 2.0.6 in /usr/share - other than that it worked fine.
One final tip if you try and restore from Time Capsule/Time Machine; when you reinstall Leopard don't create a user account with the same ID as the one you are gonna restore - as the Migration Assistant forced me to rename the restored one (which caused some problems that took me a little while and some 'chown'ing to fix :).
Other than those little gremlins; nice job Apple! If I ever have another catastrophic failure, the next restore should be pretty painless
I'm liking JAX-RS and Jersey
I'm loving the simple JAX-RS POJO based programming model (quite Rails-ish) for writing controllers and models which work beautifully in a hi-RESTful way which also now support a rails-like way of implicit templates via JSP/Velocity/whatever with a minimal amount of fuss.
I'm liking the controller-is-the-model-facade option with implicit MVC too; so the model is totally decoupled from the web and the controller just has a few annotations to deal with binding methods nicely to the URI/REST/content types and so forth - then you can have as many representations of a model as you want - just write views (e.g. index.jsp, detail.jsp, brief.jsp or whatever). Lovely and Rails-ish
Its now pretty easy to write hi-rest web apps in Java which render content as (X)HTML, XML, JSON with nice RESTful content negotiation and using good URIs.
Wednesday 14 May 2008
Apache ActiveMQ 5.1 and Apache ServiceMix Kernel 1.0-m3 Released!
Monday 12 May 2008
Squawk (simple queues using awk)
Monday 21 April 2008
Getting hooked on twitter
Friday 18 April 2008
Google AppEngine is very impressive!
Sure Amazon has lots of similar stuff at the bits and bytes level. EC2, S3, SimpleDB and the new file system stuff are cool too, don't get me wrong. For some things the power and flexibility of the Amazon offerings are great.
I guess some web developers are gonna push back on the use of python but I'm sure google will release Java / JavaScript flavours of AppEngine soon making it not much of an issue. Who knows maybe even PHP too - not sure about Ruby though, I dunno if they've figured out how to make a ruby sandbox yet.
I'm sure ultimately this is gonna be a big game changer for those making public web applications; it takes so much hassle out of making web apps - and makes creating highly scalable & available web apps very rapid & fun. Well done googlers; I'm very impressed.
Wednesday 16 April 2008
Latrz - a handy web app for reading stuff later (latrz)
I've often kept open loads of FireFox windows for stuff I should read soon (which then means I struggle to find what I am currently reading/working on from stuff I wanna read later). Then I've been through phases of cutting and pasting interesting articles into files/wikis for later. I've always felt this was sucky and wanted a nicer solution.
Now there's latrz and I confess to being totally hooked already :).
Basically if I'm on a page I know I wanna read, I click the Read Latrz bookmarket, then close the browser or carry on doing what I'm doing. Then when I've time I just click on the latrz site to read whatever takes my fancy, then mark it as read when I'm done to remove it from my reading list.
Go on, give it a try or subscribe to the Latrz Blog, you know you want to :)
Using Scala to create a better Camel DSL for Enterprise Integration Patterns
I'd previously experimented with Groovy and Ruby DSLs and found them to be little more verbose (e.g. having issues of needing to pass 2 closures/blocks to methods like when or filter, one for the predicate and one for the block to execute if its true).
You can grab his slides from the recent ApacheCon talk. I'm really liking it. The Scala use of separate syntax for functions/predicates and blocks helps improve the Camel DSL hugely
Thursday 10 April 2008
Apache Camel 1.3.0 released with 208 new features & improvements
Its taken a while to get there but we've finally got the new Camel 1.3.0 Release out of the door which includes 208 new features, improvements and bug fixes described below. Also as David has blogged recently we've MSMQ support now along with Esper and an Esper Demo. Many thanks to all those who helped ride this beast out the door. Enjoy!
- improved Bean Integration
- improved testing via Spring Testing along with the Mock, Test, DataSet components
- browsing of endpoints via the BrowsableEndpoint with support in the JMS, List, Mock, SEDA and VM components
- improved support for message exchange patterns such as InOut for JMS
- various improvements in CXF, iBatis, Jetty, MINA components
- improved XPath and XQuery support in Spring XML and with easier namespace configuration
- default to more efficient JMS usage when working with Spring 2.5.x to avoid a previous Spring bug
- various improvements in the DSL and with error handling
- improved automatic Type Converter implementations
- method invocations now supported in EL expressions
- CXF transport API now supported in CXF component, you could leverage the power of Camel mediation in Apache CXF
New Components
- AMQP
- DataSet for easier load testing
- JCR for JSR 170 support
- List for UI and tooling integration
- Stream for working with input/output streams
- Test for easier functional testing
- XQuery for easy XQuery based transforms for Templating
New Data Formats
New Languages
New Enterprise Integration Patterns
Please see Release Notes for more details. Download it now while is hot to trot!
Thursday 27 March 2008
[LazyWeb] a maven plugin to find resources on your maven dependency path
mvn classpath:find -Dresource=log4j.properties
Tuesday 18 March 2008
Writing unit tests? Give hamcrest a try
assertThat! :)<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-all</artifactId>
<version>1.1</version>
</dependency>
Friday 14 March 2008
If you are tinkering with VMWare on OS X
Friday 7 March 2008
UFace getting hotter
Friday 15 February 2008
Easier Integration & Pattern testing with Camel and Spring 2.5
I've just created a little document to show how to use Spring Testing with Camel, using the Camel Mock and Test endpoints for easier Enterprise Integration Pattern based testing. Incidentally Camel now has a new little Test endpoint which creates a Mock endpoint that automatically pulls its expected message bodies from another endpoint and auto-wires up the expectations for easier testing with minimal coding.
Public Training on Apache ActiveMQ and Apache ServiceMix
- London,UK - March 11-14
- Waltham, MA - March 17-20
Wednesday 6 February 2008
Using Apache Camel, ActiveMQ and Esper for Complex Event Processing
Update: David bas blogged about this too :)
Monday 4 February 2008
rest-ws looks great
Monday 21 January 2008
[LazyWeb] using Subversion as a front end to Confluence?
I wonder how hard it'd be to sync Confluence content with a subversion repo; so folks could still use the online Confluence site to edit content, which would get mirrored to an svn repo as *.wiki files - then folks could commit to the svn repo to update the confluence database? Anyone ever tried?
Friday 18 January 2008
bored of your OS X desktop? Try DeskLickr
Thursday 17 January 2008
Quick review of Leopard, the latest OS X
I love the new look and Cover Flow on Finder; don't much use the Dock as I use QuickSilver so the Stacks stuff doesn't get used.
I've gotta wait for me to buy a TimeCapsule before I can try out TimeMachine which does look cool. Apple are so good at sucking money out of my pocket :). Am gonna have to buy an Apple TV now too.... Wonder how long before they'll support the UK on movie rentals.
So far I still prefer iTerm to Terminal due to it being able to have preconfigured named shells & starting directories (I've often got tons of shells open).
The upgrade was great; I just installed the DVD, kicked off the install and came back later and it was all completed.
So far the only real nit was bash shells lost their prompt, so I had to add this to my .bashrc
export PS1="\w> "Not an amazing upgrade or anything, but I am preferring Leopard.
Update : I've just noticed TextEdit now opens OpenDoc files natively - nice!