I ran into the need to change a project I am working on from keeping a ton of java objects in the air due to real-time calculations and performance.  Didn’t need (or want) full database that would need to be maintained by a database group.

Every time the application was restarted, the entire thing would be reloaded based on other sources (other databases of data — all different db’s, data and transformation needed).  The transformation requires each base source to have different set of transformations and calculations performed on them.  Also since the sources have different real-time intervals there needed to be different levels or transformation going on.

Of course if I was a dba, I would be saying this should all be done in some huge db iron and storing it into some jumbo sized tables and all could be done with scripts, triggers, etc.  But I am not a dba, I like my logic in code, not a database.  That is just who I am.  I love Java and like to see what my code is doing to the data and db are used to store and get things for me, period.   Just wasn’t going down that road.  We can agree to disagree or not.

Ruled out in like 5 seconds that the use of flat files to store the calculated data would just not work.  I had to have the ability to retrieve the data fast (wanted hash maps or trees in memory, but that quickly ran out of steam due to quantity of data).  I also wanted to keep everything in the single app (single deployed war) at the end of the day.  Makes support easier in the long run.  One point to look at instead of 12!

So I came up with a list of what I wanted:

1. Needed to be quick.
2. Needed to be embeddable (not an Oracle, MySQL or anything like that) / memory friendly.
3. Needed to deploy when the web application deployed (all self contained).
4. Needed it to be support by ORM (Apache Cayenne) since already using in the project for other db work.
5. Needed to be open source.
6. Needed to be temporary, last only while web app was up and running and throw everything away when app restarted.

I had narrowed my list down to three choices I was entertaining (your experience might be different):

• Apache Derby aka Sun Java DB
• HSQLDB
• H2

They probably would all get the job down.  But I tried to narrow it down by my criteria by doing a little reading (Google + reviews + blogs + groups complaining about stuff + did I like what I saw):

1. Started reading, it appeared H2, then Derby (latest) and then HSQLDB was the speed arrangement. (YMMV)
2. More reading… All would meet the need, but H2 had some really nice instant up and running features that I liked. (IMO)
3. Same as #2, they all would meet the need more or less.  Just different feature set, but once again H2 was drawing my attention. (IMO)
4. All had native support in Cayenne since I am using 3.0 (H2 was not supported prior to 3.0, but that didn’t matter since I am using 3.0).
5. All met this need and would of been fine.
6. All met this need in some fashion and would of been fine as well.

So it really came down to how easy “did it appear” for me to get this started, embedded and actually working, using it in actual live code all in the same day.  For me it had to be H2.

There were quite a few reasons other than meeting my 6 requirements above.  I also liked that it had an active change log which every month appeared to have forward progress and making the project better.  Open source projects are either constantly evolving and or dead.  Get worried when a project hasn’t had commits or quite a bit of progress in months, 6 months, last year, etc.  Code is never perfect, it can always be better.  Don’t leave it there to just die.

I have to admit that venturing down the H2 path in the first day of use has turned out to be a great choice.

Love the embedded console you get by including 1 (one, yes one) jar in your project which you were going to do anyways since that is the only lib required.  Edit your apps web.xml and bang.  That is supporting your dev project front and back.  I can see it will come in handy trouble shooting the application both during development and even possibly productional since we are talking ever changing live data.

Integrated really quickly with Cayenne.  I have cayenne actually building the schema from its own XML each and every time the app starts up.  Talk about reuse and single place for describing the model.  ORM creates the model, creates the Java classes and now builds me my schema in my temp db.  That is a win, win, win situation folks.  No SQL monkey business, period!

With a little more code (I mean little) I have it dropping everything (entire schema) and db temp files when it shuts down (all through a single listener config in the web.xml).  I love when things are simple and just work.   Kudos to Thomas for creating such a great db that works right out of the gate.  Simplest Java library I think I have ever used.

I plan to write up some of the things I find interesting in the coming month or two (along with some other Wicket and Cayenne stuff I have mocked up, but not typed into here).   Here are some of those ideas I have already worked through turned out beautifully (eluded to above)!

• DB embedded in app config.
• Cayenne used to build H2 schema.
• Purge db on shutdown.
• Performance / usage.

It should be noted that H2 is developed by Thomas Mueller who was the creator of HSQLDB (H2 can be seen as HSQLDB2, just better and it even said that it could merge back into HSQLDB)… anyhow.

H2 +1 in my book.

It’s always interesting to see what other developers use as their real workspace when they have control over it.  (Unlike cubical heck).

Once such sweet space is something Mitch Haile built.  I stumbled across it in one of me RSS feed consumption extravaganas and have to admit the space is really, really nice, but I have to say the screen real estate and mini server room/closet is very nice for a home build.

Just had to share the nerd eye candy for others to consume!

http://www.biscade.com/office/

Snaps on Flickr:

http://www.flickr.com/photos/bwa32/sets/72157607284828899/

Very nice work Mitch!  Kudos

Well, I finally had to bite the bullet and migrate a large Wicket project from 1.3 to 1.4.  I was really not looking forward to it because of all the generic mess and model changes.

This was nothing against the Apache Wicket team, since they do an incredible job on the BEST Java framework ever (it is more than that.. but that is further down).

I just thought of having to review so much code and updating the ever growing set of Wicket “component” libraries, but that was the actual reason for the surge of change.

Some of the key sub projects I was leveraging were all converted (for sometime now) and were really starting to not go back to their 1.3x branches to update the older code.  I saw the 1.3x branches dying.  NO!!!!

I was trying to keep some of my key sub component / libraries manually synced from their newer bigger brother 1.4 code trees, but that was getting ever increasing painful and foolish.  Not to mention to find bug issues fixed in the 1.4 branches, but not in the 1.3x code base I was using.   So the decision was to maintain old branches for the sub projects (not my code even) or update my project and use all new 1.4 code for everything.

Jump… 1.4            1.4 is a very good great thing.

I know in the Wicket community there has been some huge Generic battles and long, long, long threads on it was the wrong approach, too verbose, too much code for no bang, blah, blah, blah. It was painful to watch since last year (thankfully they took it out of the forum and gave it a wiki home for folks to die over).

I actually find so far that 9 out of 10 times, it has made my code more understandable and actually working with models even more concise.  You get the added benefit of types collisions be detected in your IDE (if you use a real one) / compile time on models in the code that you would not get until you actually put the wrong object in a model for a component… then crash and burn and do a wtf was I thinking!  Nothing beats detecting issues before ever running a line of code.  I have C (not ++ or #, C) flashbacks of trying to figure out what code didn’t work, but would compile.  Boy I am glad I don’t code in that world anymore.

I am glad though, that I did hold out past the 1.4M3 period.  Much of the early on back and forth over how to implement generics was not pretty and I could only think of moving code from M1, M2 to M3, to RC…   Even sweet utility classes (albiet small classes) like ListModel and MapModel really do ease the coding and sets up the enhanced model design even better.   Once again, it makes the code more understandable as to what the Model contains and what you are working on.

Of course with any cleaning, updating has spurred on some good old refactoring of my own to leverage some new efforts in 1.4 branch and even how I plan to continue to leverage Apache Cayenne in some more elegant base Model designs of my own.  Should help with the whole deteaching and seperation level between Cayenne and Detachable models in Wicket so much cleaner.

So the dread, regret and final excitement was all part of my 1.4 migration.  At this point all projects will of course be 1.4x and some day 1.5… (Ajax enhancements could be even more awesome to come.)

Once again, it just goes to show that Wicket never steers me wrong and increases my I love for the whole framework project (99.9% of the time) that much better.  It makes Java Web development fun and exciting.  Come on kids, drink the cool aid, it won’t hurt ya!

SIDENOTE:

Wicket is not just a web framework, but a Java web way of life.   I don’t how many times I have told the story of that I will not / never (never say never right) go back to doing JSP development period.  It is just wrong once you see the light.

Coding Horror has a sweet write up on Coding Buddy.

But the part I loved the best when it showed up in the RSS feed was the associated sketch.

Priceless

I know on several projects I have ran into this and even though in certain situation you can’t blurt out WTF, but you sure are thinking it along with the slow dull screams of WTF am I doing on this project!

Coding Buddy, its truly fact and not a work of fiction.  It can make a big difference.  I know even a conference call can help shake out the WLF/minute level.  Being someone that is working on three projects (day job and other) coding buddy is solely missed.

Sometimes you don’t get the added benefit of sitting next to other developers all day.  Sometimes that is good, sometimes it makes you review some of you project code months later and the WTF meter is pegged.  What the heck was going on then.. normally deadlines that “others” wanted met, 60 days before the project even started.  I count myself lucky that in all my development life I have only been in one project that I would call a complete failure and that ship just sank of horrible death.  Definatly no coding buddies there and a bunch of developers all on islands to themselves.  Never good.

Design first, Think often, Refactor more AND talk to a coding buddy (even if it is your childhood furry bear) so the w/min levels are under control.

Friends don’t let friends code without a buddy.

The best web dev tool there is reaches 1.3.0 Final.

They have really made some great progress lately and for those of you not using Firefox 3.1 yet, you can go get this now.

Firebug 1.3.0 Final Released

If you are using Firefox 3.1 you will still need to go get the 1.4 alpha (or soon to be beta).

Firebug Release Directory

Thank you again Firebug team!

There is a great Coding Horror write up by Jeff which gives a interesting perspective (most likely management.. but still the same) on the trade off of what your developer costs you and what new more powerful hardware costs you.

Hardware is Cheap, Programmers are Expensive

The post eludes to several key points, many of which I try to get management on numerous efforts to understand all the time.  Some hold true, but there is a point of no return.  Sometimes just throwing iron at it will only make more of mess (especially in the energy conscious era we have stepped into these recent years.)

Remember the quote, “To make money, you have to spend money.”… well the spin I see here is, “To save money, you have to spend money, sometimes.”

This directly ties into the The Programmer’s Bill of Rights.

I remember the days of Windows development that I heard that they did not want to give the development team too fast of machines, since then the code they wrote would never perform well on “normal” hardware that end users would have.  That was some what true, but much less important (excluding fancy fancy stuff) in a web centric world.  Now it has to deal with the size of the pipe and not the engine.

Over the years I have seen more times than not, is not about optimizing (as eluded to by Jeff) but cleaning and/or refactoring.  I usually find 9 out of 10 times if I refactor code when trying to make more out of it I get more bang for the buck.  But that also means you live a life of refactoring (which all good developer should).  This helps with:

• Stop the code decay / erosion.  All code dies over time.  (Try to get management to understand that one right?  Some developers don’t even understand it sadly enough.)
• Make the code more clear and concise for future developers who will have to read it.  (If it doesn’t you probably refactored wrong).
• Enhance the current functionality with less code (more bang for the buck).  Yes Virgina, there is a Santa Claus and this can happen to you too.
• Evolve the code to newer library/standards/technology.

The last bullet point is something I have found very important in the Java world.  You find that you have (or someone else) has written code for a project 5,6,10 years ago based upon JDK, lib, etc at the time.  If you evolve the code to use more modern JDK run-time libraries or even specific 3rd party libs/projects that didn’t exist back then, you can get some great windfall.  (Of course done with great scrutiny).

As always, there is not just black and white.  There is not just one solution to all problems.  Find the right shoe and make sure it fits.