Humphrey Sheil

NoSQL / NewSQL / SQL - future-proofing your persistence architecture (part one)

2011-08-07T22:59:00.001+01:00

Although its been a few years in the making, the noise / buzz around NoSQL has now reached fever pitch. Or to be more precise, the promise of something better / faster / cheaper / more scalable than standard RDBMSs has sucked in a lot of people (plus getting to use MapReduce in an application even if it's not needed is a temptation very hard to resist..). And pretty recently, the persistence hydra has grown another head - NewSQL. NewSQL adherents essentially believe that NoSQL is a design pig and that a better approach is to fix relational databases. In turn, NewSQL claims have been open to counter-claim on the constraints inherent in the NewSQL approach. It's all very fascinating (props for working Lady Gaga into a technical article as well..).

As it turns out, traditional RDBMSs are sometimes slow for valid reasons, and while you can certainly speed things up by relaxing constraints or optimising heavily for a specific use case, that's not a panacea or global solution to the problem of a generic, fast way to store and access structured data. On the other hand, the assertion that Oracle, MySQL and SQL Server have become fat and inefficient because of backwards compatibility requirements definitely strikes a chord with me personally.

The sheer variety of NoSQL candidates (this web page lists ~122!) is evidence that the space is still immature. I don't have a problem with that (every technology goes through the same cycle), but it does raise one nasty problem: what happens if you back the wrong candidate now in 2012 that has disappeared in 2015?

The current NoSQL marketplace demands a defensive architecture approach - it's reasonable to expect that over the next three years some promising current candidates will lose momentum and support, others will merge and still others will be bought up by a commercial RDBMS vendor, and become quite costly to license.

What we need is a good, implementation-independent abstraction layer to model the reading and writing from and to a NoSQL store. No hard coding of specific implementation details into multiple layers of your application - instead segregate that reading and writing code into a layer that is written with change in mind - we're talking about pluggable modules, sensible use of interfaces and design patterns to make the replacement of your current NoSQL squeeze as low-pain as possible if and when that replacement is ever needed.

If the future shows that the current trade-offs made in the NoSQL space (roughly summed up as - a weaker take on A(tomicity),C(onsistency), I(solation) or D(urability), plus with your own favourite blend of Brewer's CAP theorem) are rendered unnecessary by software and hardware advances (as is very likely to be the case), then the API should ideally insulate our application code from this change.

There are interesting moves afoot that demonstrate that the community is actively thinking about this, specifically the very recent announcement ) of UnQL (the NoSQL equivalent to SQL - i.e. a unified NoSQL Query Language). That's good, but UnQL is young enough to shrivel and die just like any of the NoSQL implementations themselves. Also, we know that what has inspired UnQL - SQL - is itself fragmented / with vendor-specific extensions like T-SQL from Microsoft and PL/SQL from Oracle.

So then, in part one of this two-parter, I've worked to justify what's coming in part two - a minimal set of Java classes and interfaces to provide a concrete implementation of the abstract ideas discussed above.

New Google Analytics location report doesn't like Connacht so much..

2011-07-31T19:27:00.000+01:00

The new UI for Google Analytics has a distinctly Cromwellian vibe to it, as the screenshot below shows. Is this just my GA account, or does everyone else see Galway and Sligo a bit more surrounded by the Atlantic than normal?

Umbraco on Azure - take 1.5 (not 2)!

2011-07-06T14:24:00.001+01:00

Back in August of last year I wrote a step-by-step article on how to get Umbraco running on Windows Azure (the Microsoft cloud computing platform). It got a lot of hits from people looking to do just exactly that.

There were a few loose ends in that piece, notably not using shared rather than VM-local storage to allow for Umbraco clustering and also not using the .NET 4.0 runtime rather than .NET 3.5 (4.0 was a recent addition to Azure in Aug 2010 and it just didn't work out of the box - missing sections in the machine.config).

So a follow-up article has been on my to-do list for a while now to tie up these loose ends, and then I found this - the Umbraco Accelerator for Windows Azure.

I have no idea how good / bad it is, but it's a great idea and well worth a road test if you're looking to use Umbraco in production with Azure.

Seems to be still active since it's initial release in Oct 2010 with a point release put out there in mid-June.

It also appears to be part of a wider plan to standardise how ASP.NET applications can be moved to Azure in a standard way (the Windows Azure Accelerators project), again a good thing IMHO.

Let me know how it works for you.

Google IO 2011 Day Two recap

2011-05-14T02:38:00.000+01:00

Oh the perils of making predictions when there is still a conference keynote to go!

It turns out that Chrome OS and the associated hardware hasn't been read the last rites after all. Rather, v1.0 is almost ready for primetime (scheduled for release in mid-June - about a month away). You have to imagine over time though that Google will want one code base for phones, tablets and chromebooks. At the very least, they will want to make it as easy as possible for developers to write their applications once and have them "just work" on devices with radically different screen sizes and input methods, something that Android developers today are already doing. Nonetheless, a very brave play, especially in targeting the enteprise space, where significant replacement costs exist. If it pays off, it will be huge.

Moving on from Chrome, a couple of sessions I attended yesterday were really interesting, specifically two - Full Text Search and Smart App Design.

Full Text Search is Google's take on Lucene / Solr and integrated into the App Engine Datastore as well, so it will be compelling for developers who just want to start indexing and scoring documents quickly. The "fully automatic" mode of operation with the Datastore should also be a timesaver.

Smart App Design covered material of a completely different color. I had already read about the Prediction API in the blogosphere but I hadn't realised exactly what it did until this session. Essentially, Google offers the discerning developer the ability to add machine learning techniques to their application by leveraging a cloud-based service.

At first glance, I had thought that the API gave access to the same model that Google uses to predict search terms, and I guess that is one use case. But Google has done much more than that - they have effectively white-labelled their machine learning technology and made it available to non-Google developers to use with their own data, i.e. learn what's important for their application / business.

As with all machine-learning techniques, the nub of the matter remains the correct selection and efficient representation of the key attributes in the training set, and that is quite simply a problem that requires deep domain knowledge. One announcement yesterday was quite interesting however, in that Google are now allowing good model authors to sell their models to others. So if I come up with a model that predicts shopping basket behavior on leisure travel websites and a tour operator used that to bump their online conversion rate by 33%, then that model has a lot of value and it's a win-win situation for the model author and the model user.

So an API with a lot of promise. But also with two potential flies in the ointment, one commercial and one cultural:

(a) Commercial - Google are trying to charge for use of the API from day one, this will stymie adoption in the earliest stage

(b) Cultural - an endemic problem with a lot of machine learning techniques is their black box nature. As someone who spent a fair bit of time working with artificial neural networks at university, quite often a machine learning approach will yield the correct answer but the researcher can't exactly explain why! That's not a Google-specific weakness, but what is Google-specific is that the modules you access via the Prediction API (the man behind the curtain if you will) is not made open at all, so can a company really invest time in building, training and using models that they don't really understand and can never hope to do so? Only time will tell.

So to recap then, Google IO was definitely worth attending this year - and not just for the hardware gifts! The main items on my research list post the event are:

1. Google Go running on App Engine

2. The Prediction API

3. Full Text Search enhancements / module for App Engine

4. Adding my own hooks and content into Google Maps and Street View to greatly enhance what the end user sees when they access Maps from my site

5. Fusion tables + Charting - a good / cheap way to rapidly slice and dice data and provide good interactive widgets to visualize same to end users.

Google IO 2011 (#io2011) - day one recap

2011-05-11T05:36:00.001+01:00

The official Google code site has the lowdown on all of the announcements that came thick and fast today (some 11 major items last time I checked and plenty of API revs and upgrades) and I won't replay them all here.

Specific announcements that interested me today:

Google Go is about to become an officially supported language on App Engine, alongside Python and Java (it's currently in "Trusted Tester" mode).

Rhetorical question: what value does a complete end-to-end technology stack with no overhanging IPR issues or blockers have to Google as a potential insurance policy in case the Oracle lawsuit does not go in their favor / be settled reasonably? Two things I heard today convinced me that there is now serious engineering investment going into Go (as opposed to a small, talented team cranking things out as they work down the list):

(a) The afore-mentioned App Engine support (this won't have been trivial to implement - Go is the first compiled language to run on App Engine after all for one thing)

(b) The info that a "comprehensive" Go library for ultimately all of the Google APIs is in development and will be with us "soon".

Go is a very nice language to write in, and the App Engine support announced today addresses one of the major gaps I identified when I took a look at Go when it was first released in Nov 2009.

Three final comments on day one:

1. Press articles I read in March / April this year about the +1 button being a make or break deal for Google to compete with Facebook seem overblown. The +1 button has merited just one session so far and apart from that you wouldn't even know Google had it. Either that or the memo didn't make it to the IO organisers in time.

2. It's instructive to watch Google see the mistake that companies like Sun Microsystems made and impressive to watch how they studiously avoid it. It's not enough to develop great code / software / hardware - you have to have people **using** it. Google's continued push into content ensures that usage. Google is not just the place you go to find content on the web, it's also where you consume that content (first youtube, but now books, movies and music too). I'm glad Google don't have a social network offering in their portfolio of services - they would be simply too powerful if they did.

3. Google IO seems to be **all** about Android so far - it's absolutely everywhere you look and consumed the entire keynote this morning (Ice Cream in Q4 that unifies tablet and phone, Futures (Android @ Home), open accessories etc.). Barring some crazy and unforeseen announcement tomorrow, I'd say Chrome OS has been given the last rites internally. But then again, who knows what day two will bring?

A vision for big data in leisure travel ecommerce

2011-03-06T20:10:00.000Z

[This is an article for people working in leisure travel technology / ecommerce online conversion who visit this blog, although many of the take-home points are transferable to other industry verticals.]

Data is big, and getting bigger. The more we track and log, the more storage is needed to warehouse it, and the more CPU horsepower is needed to mine it to answer questions posed by the business. As an aside, everyone is facing this issue and it's sink or swim, with the swimmers sure to get a competitive advantage over the sinkers. In this article, I'll examine the main data feeds that matter in leisure travel, and propose an architecture to collect, manage and mine them for business benefit. The end goal is to propose a vision, explaining why and how to collect data to better inform and drive business decisions that improve ecommerce performance.

But why now - hasn't this always been an issue? Yes, but now more than ever, leisure travel is poised on the cusp of another big game-changer. Companies like Google and Microsoft are clearly already focusing more on travel as a segment, and their data gathering and mining capabilities are considerable. But tour operators and online travel agencies (OTAs) have a significant competitive advantage over pure play technology companies as we'll see a little later.

Important data sources in leisure travel ecommerce

First, let's examine the primary data sources that affect leisure travel ecommerce. There are some obvious entries in the table that follows, and some less so.

ID	Name	Internal / External	Controllable	Purpose / Comment
1	Availability (internal)	Internal	Yes	Stock (internal, at-risk / committed inventory) available to sell, down to room type / meal plan / cabin and fare class
2	Pricing (internal)	Internal	Yes	Pricing for internal stock. Entire teams stay focused on this source, ensuring it is (a) competitive, and (b) profitable
3	Availability (external)	External	Yes	Stock available to sell contracted through third parties (usually not committed stock), down to room type / meal plan / cabin and fare class. Usually used to plug gaps in internal stock (resort coverage, star rating, price band etc.). Sources include GDSs, bed banks, car rental companies etc.
4	Pricing (external)	External	Yes	Pricing for third party stock.
5	Rich content (internal)	Internal	Yes	Provide compelling, unique, accurate text, images and video to convince the consumer to buy
6	Rich content (external)	External	Usually	Provide compelling, unique, accurate text, images and video to convince the consumer to buy. Needs to be differentiated otherwise your search engine ranking score will suffer due to duplicate content penalties.
7	Attributes	Both	Yes	Attributes (aka facets) are becoming increasingly important - star rating, price bands, family-friendly (has a creche, rooms are adjoining), "has a", "is a", "is close to" - attributes provide consumers with a more intelligent and targeted search capability
8	User generated content	External	No	Tripadvisor is the poster child here, but user generated content (UGC) can be in-house too - but it must be perceived as unbiased by the consumer, otherwise it becomes a negative.
9	Meta data	Both	Yes	Every business tags its own data - timestamps, version numbers, # revisions, author, approver, when last yielded. The more meta data you have the merrier - it often helps to tie disparate data sources together and enriches the overall data pool
10	Search, cost, book funnel	Internal	Yes	Traditionally the core of any ecommerce strategy - measures the complete search, cost and book journey. Needs to be fully instrumented to collect data so that A/B and multivariate testing can be used to fine-tune performance over time. Google Analytics does this very, very well.
11	Offline (shop) interactions	Internal	Yes	Few businesses try to tie shop activity back to online activity, but for a bricks and mortar plus clicks business, this is an opportunity missed
12	Online advertising (SEO)	Internal	Partially	SEO can be thought of as PPC you don't pay for! Critical to making cost of acquisition online as efficient as possible. Only partially controllable due to businesses being at the mercy of search engine scoring (which both Google and Microsoft (Bing) keep as a black box algorithm)
13	Online advertising (PPC)	Internal	Yes	Where Google makes its money!.. PPC has pride of place in every well-constructed ecommerce campaign, but the cost and effectiveness should be continuously monitored, challenged and tuned. CSV exports out of AdWords provide a good way to do this
14	Personalisation	Internal	Yes	Personalisation - both anonymous and known, is a great way to learn what kind of holiday / vacation people want to buy from you and how they want to find and buy it. Just don't try to build personalisation before you have (10) working well - personalisation needs a really solid foundation to work well..
15	Social media	External	No	The rising star that no-one really knows how to handle. The Facebook API contains a lot of potential for travel ecommerce
16	Offline / traditional advertising	External	Yes	The efficacy (or not) of ad spend must extend to traditional / offline as well as the more easily measurable online variant, otherwise you don't know where all of your marketing £s / $s / €s are going
17	Post-booking interactions	Internal	Yes	ecommerce data source, but savvy businesses are now looking at post-booking amendments, cancellation rates etc. to identify patterns that can feed back into the search experience
18	Customer Relationship Management (CRM)	Internal	Yes	Both pre and post travel - it's key to have a good view of what the customer experiences on holiday and feed that back into what holidays are sold going forward. Is that picture of the pool misleading - change it! If the service is great, promote it more!

Table 1. A proposed taxonomy on data sources that impact and influence leisure travel ecommerce.

Two important characteristics of data are whether you control it or not (and hence can change it if you need to) and whether it is sourced from an internal system or an external system (and thus how trustworthy / accurate the data is and whether it is unique to you or if other business entities can see it too). We have added these two characteristics to the table above for clarity.

What should be obvious to the reader is that a holistic picture of ecommerce performance requires multiple data sources, some of which traditionally would not be seen as impacting the effectiveness of a leisure travel ecommerce system. Gone are the days of simply looking at the web logs to see how effective (or leaky) the conversion funnel is! In fact, there are probably some sources that I've inadvertently omitted, and indeed as new systems come on stream, new sources will be added to this table / taxonomy.

Finally, it's interesting from a barrier to entry perspective to note that only the well-placed tour operator or OTA actually has the wherewithal and access to collate data from all of the sources noted in the table. Other new entrants simply do not have access to many of the sources listed. The data itself is now a valuable commodity (and is increasing in value), and an asset that leisure travel businesses would do well to guard jealously.

What we need - Systems and Data working together

At present, I contend that the average tour operator / OTA is collecting some, but not all of the data sources identified, and that no tour operator or OTA has yet constructed a system that provides a holistic, joined-up view of the data back to the business function to inform decision-making activities. Why not? Because it's not easy to do! The IT estate behind these data sources is fragmented (core res system, yielding system, multiple content management systems, external systems, separate booking repositories / agency management systems, Google Analytics, Google AdWords, Excel spreadsheets), often owned by different companies and wasn't designed to provide with the kind of view that is now needed. Ominously, new entrants into the space do not have a lot of the legacy baggage that incumbents do, meaning their velocity of implementation and ongoing change creates a hard-to-ignore imperative for all sellers of leisure travel to innovate quickly and learn from their data, or be left behind.

The technical challenge is four-fold:

1. Collection and storage - gather and store as much data as possible for each data source in the table, with that data being as clean and structured as possible (and in the real world, every data set will have some noise to it)

2. Build a holistic, joined-up data set - identify ways to link the data sources together - version number, unique keys, foreign keys, link backs, tagging etc. The more your data sources are joined up, the more holistic a view of the business you are building (and can provide back to the business). Conversely, disconnected data sets (data islands) are of much less value to the business and introduce the risk of an incomplete / inaccurate view of what's really happening now being used to influence what's going to happen next

3. Answering the questions - provide a mechanism to answer questions over this corpus of data in near real-time to allow the business to modify its behaviour and focus to maximise profits, yield and margin

4. Suggesting the questions - once the above three points have been implemented to a mature and repeatable level, the final logical step is for the data function to actually suggest areas of improvement and further exploration based on emergent patterns in the data, using techniques such as artificial neural network and self-organising maps (SOM) analysis

Putting it all together - a suggested framework

There are many ways to construct a view over the data sources identified in the previous section. And in fact, multiple views are encouraged depending on the goal of the business. Here however, a hybrid of time and business function is selected in order to select a reasonable framework to hold the data. This framework is depicted in the following diagram.

Figure 1. High-level schematic of the big data system for leisure travel ecommerce.

A concrete implementation of the framework

The question naturally arises - how would this system be constructed, not just initially but also maintained and extended going forward?

Some natural candidates already exist, chief among them Cassandra and Hadoop. In the author's opinion, a hybrid architecture of Cassandra's data storage and innate simplicity and high availability, coupled with the MapReduce framework from Hadoop offers the best blend of performance, scalability, availability / resilience, querying and extensibility. A separate follow-on instalment to this article is warranted to provide a detailed technical treatise on the underpinnings of the system outlined here.

Conclusion

The dominant data sources that impact the effectiveness of a leisure travel ecommerce strategy are identified, named and classified. Developing this classification further, a model is used to create a framework to house the data sources and a concrete implementation suggested.

About the author: Humphrey is the Chief Technology Officer for Comtec Group, a company that specializes in leisure travel technology.

JDK 7 preview and JEE 7 planning

2011-03-02T20:44:00.000Z

We got two interesting developments in Java land this week:

1. Oracle released the developer preview of the Java 7 Development Kit (JDK)

2. Oracle have started talking publically about what JEE 7 (and beyond - JEE 8) will look like in Q3 2012 and Q4 2013.

(1) has been a long time coming and it's good to see the log jam moving. Simply shipping JDK 7 is good in its own right but it also means that the team will move onto working on JDK 8, which contains some key language features omitted from JDK 7 so that the team could JGIOTFD (Just Get It Out The (reader exercise to complete the acronym)).

(2) looks to be Oracle really making the JEE stack cloud-based / cloud-friendly by default rather than a technology stack that merely facilitates cloud computing. This dynamic should see Oracle formalising exactly what constitutes "JEE in the cloud" via a JSR and thus wresting that intellectual responsibility back from Google's App Engine platform, which is pretty much the de facto standard for "JEE in the cloud" at present.

Looking beyond JEE 7, JEE 8 looks to be embracing Big Data / NoSQL systems like Hadoop and Cassandra, although we can expect to have seen significant consolidation in this space by 2013, making the integration and platform support task easier to accomplish.

All in all, two nice moves, and good news for the Java eco system / economy. You might or might not like Oracle, but they are getting stuff out the door in a way that Sun kind of forgot how to do.

Oracle Certified Enterprise Architect - JEE 6 refresh update

2011-02-22T08:38:00.000Z

The JEE 6 SCEA exam / certification

Following on from my earlier post requesting input into my next post, here's an often-requested update: what's happening with the Oracle Certified Master, Java EE 5 Enterprise Architect exam / certification update to JEE 6 standard?

In a nutshell, here's where it is (covering each of the three parts in turn):

Parts two and three of the exam (the practical elements) will remain very similar to how they operate today - these elements test your ability to design and document (part two) a solution to a well-defined business problem using the JEE platform and then challenge you (part three) to self-critique and justify key design decisions taken, especially on how non-functional requirements will be adequately satisfied. Parts two and three are pretty much independent of the current JEE revision, because the candidate is given a good degree of latitude in how you use JEE to solve the problem. Were you to use J2EE 1.4 features let's say, then the examiner is going to question the logic of that decision closely, but that's about it. Writing Ruby code and then having it compile to Java bytecodes at runtime using JRuby is also not recommended (don't laugh, someone did ask..)!

Part one of the exam (the multiple-choice exam) **will** change for JEE 6 - it has to because part one is more tightly coupled to a specific JEE revision - currently JEE 5 (with ~5% of J2EE 1.4 content).

The last time we revised part one, ~ten architects got together in Broomfield, CO for a week to design and critique the corpus of questions used. After that, Sun Microsystems (as they were then), brought in some external testing folks to benchmark the exam and to critique the overall marking strategy we intended to employ. That was an intense week and overall a fairly involved process, because you want to write difficult, tricky questions that will challenge an architect but at the same time, be fair. Part one of the architect exam is also not allowed to test your ability to memorize APIs or specifications - that is the primary task for the lower certifications. You very quickly find that a lot of difficult / tricky questions in JEE revolve around the APIs and specifications!

I think with the benefit of hindsight we erred on the side of fairness over toughness. I think we'll look to toughen up the questions for JEE 6.

I don't expect Oracle to reconvene the team of architects to do this refresh - the last refresh of the exam was a major refresh whereas we would consider this refresh to be more minor. Therefore the time taken to update should be shorter. Once the part one refresh is scheduled in, I'll post again on this topic. For now, the JEE 5 architect exam remains the most current and up to date architect exam you can take.

What would you like to read about next?

2011-01-23T20:49:00.000Z

I've been pondering what next to write about and thought - why not ask the readers? So here's your chance!

Turns out that most people visiting / watching this blog fall into four camps (in no order of priority):

* Want to know more about Enterprise Java architecture / software architecture in general

* Want to know more about the Oracle Certified Enterprise Architect exam for the Java platform (I'm a co-author of the study guide for this exam as well as a co-lead assessor)

* Want to know more about .NET (especially running Umbraco on Windows Azure and / or MVC 3)

* Want to know more about ecommerce tracking (measuring, then improving online conversions)

At least, that's what the web tracking software gods say! There's a great mix of visitors too from all corners of the globe, but the next post will be in english I'm afraid.

So if there's a specific topic relating to the categories above that you'd really like to see covered, drop me a note at hsheilblog@gmail.com and I'll do my best to - and may the best suggestion win!

Ecommerce: online conversion - simple model and toolset

2011-01-09T21:27:00.001Z

Readers of this blog can wax lyrical on how to build a great B2C ecommerce site - either in JEE or .NET. First we get the technology stack right, then frameworks using that technology stack, comprehensive functional and technical specs, testing plans, coding standards + reviews with daily scrum meetings, hardware / cloud estimation and then load / penetration testing - this is bread and butter to the software architect.

What a lot of software architects don't understand (or underestimate) is what needs to happen to their site after it goes live. After the go-live of a B2C ecommerce site, a whole other team (which is fairly non-technical) takes it over. This team is really exercised by and focused on three core goals:

1. Get qualified visitors to the site as cost-effectively as possible

2. Enable those visitors to find the product they want quickly and easily

3. Convert the visitor into a customer - convince them to buy on your site

These goals are completely measurable in monetary terms, and hence you will find senior management taking a serious interest in them as well.

I work in leisure travel, and there are some very specific nuances to achieving these goals in my industry sector (every industry sector will have their own nuances). But there is also a generic model to be found and some very useful (and free!) tools that you can use to put the model in place.

Turns out the model is pretty simple. Essentially it consists of three components:

1. Analytics - where we measure what's happening on our target site - how is the user interacting with the site and can we infer what they do and don't like based on measuring and studying those interactions

2. Hypothesis testing (aka A/B and / or multivariate testing) - Analytics will give us lots of data to generate ideas on how to improve interactions, therefore we need a mechanism to test out hypotheses in a semi-automated way (if I change X, I bet the conversion rate will increase by Y%)

3. Efficient prospect capture - we want the best native SEO score possible on all of the search engines and when we spend money on ad campaigns, we want the best return for that investment.

So that's the high-level model - it's pretty simple.

Many companies (and especially Google), make an awful lot of money around online ecommerce. And that's where the "free!" I noted above comes in. It makes sense for Google to give away the tools enabling Analytics (1) and Hypothesis testing (2) for free, as they make so much revenue on selling ad campaigns in Efficient Prospect Capture (3). Unkind souls might claim that if you spend any kind of money with Google AdWords at all, then you're not really getting (1) or (2) for free, but you won't find a nefarious cheap shot like that on this blog.

Let's look at how we can implement the model then:

1. Analytics - use Google Analytics. Brian Clifton's book is an excellent treatise on the application, and the online training videos are of a high standard as well. It's well worth having a couple of developers on your team get Analytics certified to understand what the tool can do - it really is very powerful

2. Hypothesis (A/B, multivariate) testing - use Google Website Optimizer. There's less information about this tool, I guess because it's a bit simpler than Analytics, but a good overview is available. Being able to change content and see the impact on the fly is a key part of the model - that's why we use a CMS like Umbraco!

3. Efficient prospect capture - SEO, SEO and more SEO. The Art of SEO is a great read. My opinion here is that as long as you're doing a great job on your own SEO, you should begrudge a search engine every penny. By using tagging in conjunction with Google Analytics (make sure you associate your AdWords account with your Analytics account to get all this done for you automagically), you can continually check that your ROI on ad campaigns is worth the spend, and stop buying terms that don't make money.

And that's pretty much it. A three-component generic model for online ecommerce, followed by the simplest (with zero cost) way to implement that model for your B2C site. I intimated that each industry sector has its own quirks and foibles above and beyond this base model, and I'll focus on the leisure travel industry in more detail in a future post or two. For now, enjoy!

What I do

2010-12-28T17:33:00.003Z

(I can be contacted by email or twitter).

I'm the Chief Technology Officer for Comtec Group - we build end to end systems for clients in the leisure travel industry, primarily in the UK and US. I spend a lot of time with our architects, product managers and customers.

The CTO function at Comtec leads the definition and construction of our travel suite, from fast loading of inventory (e.g. Hotel, Air, Transfers etc.) through GDS selection and implementation and with a particular focus on ecommerce. In the ecommerce world we help our customers to measure and increase online conversion rates, optimize PPC spend, increase SEO scores and overall consumer engagement. In order to do this, we leverage analytics, A/B with multivariate testing and personalization techniques, to name just a few tools and techniques in our kit bag.

Before Comtec I worked for a financial services company as a software architect and before that again I worked as a consultant for a well-known business and IT consulting company.

In 2000, I became an external examiner and subject matter expert for the Java Enterprise Architect accreditation from Sun Microsystems - now Oracle. I have presented at JavaOne and written numerous articles on many different aspects of software engineering. In 2010, I co-authored the definitive official study guide to the SCEA exam itself.

I am deeply rooted in Computer Science - I have a particular interest in distributed systems and hold a B.Sc (1998 - First Class Honours) and M.Sc (2002) in Computer Science from University College Dublin. My M.Sc. thesis focused on building a high-throughput grid-like compute engine using Java and Artificial Neural Networks to solve a well-known bioinformatics problem (protein secondary structure prediction).

Early Xmas Cloud presents from Microsoft, Google..

2010-12-03T00:06:00.000Z

Just about 48 hours apart, Microsoft and Google have released significant updates for their Azure and App Engine cloud offerings just in time for Christmas.

The 1.4.0 App Engine SDK addresses some long-criticised weaknesses, in particular not being able to keep an instance ready to rock and roll at all times plus the ability to execute long-running requests (> ten seconds). The ole App Engine has been getting a bit of a kicking recently in the blogosphere so this is a timely release (assuming the unplanned outages have been sorted out in parallel with this). There's nothing in the release notes about a more SQL-like persistence store like SQL Azure, so you still need to wrap your head around Google's Datastore and the pros and cons it gives you.

The 1.3 Azure SDK also addresses some weaknesses in Azure, in particular now allowing developers to actually RDP onto their Azure boxen in the cloud, a really big improvement on the current state of affairs (basically you get a headless box with non-straightforward access to log files via the Windows Azure Diagnostics service).

It's interesting how these SDK releases are solidifying the differences between these two cloud offerings - Google are zeroing in on providing a PaaS model, where you have to code in a supported programming language (currently either Java or Python - wonder when they will support Google Go?) against a locked-down set of APIs, where Microsoft are moving more towards an IaaS model where you do what you like cos it's more or less your box. Both approaches have their strengths and weaknesses, the overall ecosystem is stronger for having both.

The curious case of Oracle, the JDK and plan B (aka the prune juice plan)

2010-09-27T10:39:00.002+01:00

Mark Reinhold (Chief Architect of the Java Platform Group at Oracle), posted a Plan A and B approach (just like a classic A/B ecommerce conversion test eh?!) for the JDK roadmap in advance of the annual Java love fest that is JavaOne in San Francisco last week. For me, this was the biggest item I was looking for - the time gap between JDK 6 and 7 has been ridiculous.

From his "Re-thinking JDK 7" post, the options proposed are:

<snip>

Plan A: JDK 7 (as currently defined) Mid 2012

Plan B: JDK 7 (minus Lambda, Jigsaw, and part of Coin) Mid 2011
JDK 8 (Lambda, Jigsaw, the rest of Coin, ++) Late 2012

</snip>

I am firmly in favour of the option eventually selected - option B. It's clear that the JDK has a huge feature log jam. Selecting option B is like giving the JDK release schedule a big dose of prune juice - you know something's gonna start moving.

So to understand what Plan B means for you as a Java architect, I suggest that it can be broken down into these four steps.

1. Read the negative comment to a further post by Mark announcing the decision - this comment represents why you would be unhappy with Plan B. I reproduce it here for the lazy reader (not you, the other guy):

"Hi Mark,

To me, "JDK 7 minus Lambda, Jigsaw and part of Coin" doesn't sound much like "Getting Java moving again" :-(

This schedule is very disappointing.

Posted by Cedric on September 08, 2010 at 10:06 AM PDT"

2. Read the response to the negative comment to understand what Plan B entails. Again, reproduced here:

"JDK 7 - (Lambda + Jigsaw + part of Coin) = Most of Coin + NIO.2 (JSR 203) +
InvokeDynamic (JSR 292) + "JSR 166y" (fork/join, etc.) + most everything else
on the current feature list (http://openjdk.java.net/projects/jdk7/features/) +
possibly a few additional features TBD.

Posted by Mark Reinhold on September 08, 2010 at 10:26 AM PDT"

The TBD bit is a tad ambiguous - let's ignore it by assuming nothing major is going to get in now, given the sheer volume of regression and platform testing needed before a JDK hits gold / GA status.

3. So now you know Project Coin is the biggie for JDK 7 - therefore you need to download presentation for same from this year's JavaOne 2010 session on Coin (119 slides, but a lot of these are just slides bitching about how hard it is to do, seminal slides are 10 and 23 - 66). Try-with-resources (Automatic Resource Management) looks great - equivalent to C#'s using keyword. Enhanced exception handling will enable better code as well.

4. [Optional, for the dedicated reader] Some more light bedtime reading - follow the links from the JDK 7 roadmap, especially for Project Lambda (closures) and Jigsaw (modular Java). This will then get JDK 8 on your forward-looking radar.

Now the **real** question is what will JEE 7 look like?!

Umbraco CMS - complete install on Windows Azure (the Microsoft cloud)

2010-08-29T14:50:00.015+01:00

We use the Umbraco CMS a lot at work - it's widely regarded as one of (if not the) best CMSs out there in the .NET world. We've also done quite a bit of R&D work on Microsoft Azure cloud offering and this blog post shares a bit of that knowledge (all of the other guides out there appear to focus on getting the Umbraco database running on SQL Azure, but not how to get the Umbraco server-side application itself up and running on Azure). The cool thing is that Umbraco comes up quite nicely on Azure, with only config changes needed (no code changes).

So, first let's review the toolset / platforms I used:

* Umbraco 4.5.2, built for .NET 3.5

* Latest Windows Azure Guest OS (1.5 - Release 201006-01)

* Visual Studio 2010 Professional

* Azure SDK 1.2

* SQL Express 2008 Management Studio

* .NET 3.5 sp1

Step one is simply to get Umbraco running happily in VS 2010 as a regular ASP.NET project. The steps to achieve this are well documented here. Test your work by firing up Umbraco locally, accessing the admin console and generating a bit of content (XSLTs / Macros / Documents etc.) before progressing further. (The key to working efficiently with Azure is to always have a working case to fall back on, instead of wondering what bit of your project is not cloud-friendly).

Then use these steps to make your Umbraco project "Azure-aware" . Again, test your installation by deploying to the Azure Dev Compute and Storage Fabric on your local machine and testing that Umbraco works as it should before going to production. The Azure Dev environment is by no means perfect (see below) or a true synonym for Azure Production, but it's a good check nonetheless.

Now we need to use the SQL Azure Migration Wizard tool to migrate the Umbraco SQL Express database. I used v3.3.6 (which worked fine with SQL Express contrary to some of the comments on the site) to convert the Umbraco database to its SQL Azure equivalent - the only thing the migration tool has to change is add a clustered index on one of the tables (dbo.umbracoUserLogins) as follows - everything else migrates over to SQL Azure easily:

CREATE CLUSTERED INDEX [ci_azure_fixup_dbo_umbracoUserLogins] ON [dbo].[umbracoUserLogins]

(

[userID]

)WITH (IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF)

Then create a new database in SQL Azure and re-play the script generated by AzureMW into it to create the db schema and standing data that Umbraco expects. To connect to it, you'll replace a line like this in the Umbraco web.config:

with a line like this:

<add key="umbracoDbDSN" value="server=tcp:<<youraccountname>>.database.windows.net;database=umbraco;user id=<<youruser>>@<<youraccount>>;password=<<yourpassword>>" />

So we now have the Umbraco database running in SQL Azure, and the Umbraco codebase itself wrapped using an Azure WebRole and deployed to Azure as a package. If we do this using the Visual Studio tool set, we get:

19:27:18 - Preparing...

19:27:19 - Connecting...

19:27:19 - Uploading...

19:29:48 - Creating...

19:31:12 - Starting...

19:31:52 - Initializing...

19:31:52 - Instance 0 of role umbraco452_net35 is initializing

19:38:35 - Instance 0 of role umbraco452_net35 is busy

19:40:15 - Instance 0 of role umbraco452_net35 is ready

19:40:16 - Complete.

Note the total time taken - Azure is deploying a new VM image for you when it does this, it's not just deploying a web app to IIS, so the time taken is always ~ 13 minutes, give or take. I wish it was quicker..

Final comments

If you deploy and it takes longer than ~13 minutes, then double check the common Azure gotchas. In my experience they are:

1. Missing assemblies in production - so your project runs fine on the Dev Fabric and just hangs in Production on deploy - for Umbraco you need to make sure that Copy Local is set to true for cms.dll, businesslogic.dll and of course umbraco.dll so that they get packaged up.

2. Forgetting to change the default value of DiagnosticsConnectionString in ServiceConfiguration.cscfg (by default it wants to persist to local storage which is inaccessible in production - you'll need to use an Azure storage service and update the connection string to match, e.g. your ServiceConfiguration.cscfg should look something like this:

<?xml version="1.0"?>

</ConfigurationSettings>

</Role>

</ServiceConfiguration>

You also need to run Umbraco in full-trust mode, otherwise you will get a security exception when Umbraco tries to read files that are not inside its own "local store" as defined by the .NET CAS (Code Access Security) sub system running on the production Azure VM. In other words, you need the enableNativeCodeExecution property set to true in your ServiceDefinition.csdef like so:

<?xml version="1.0" encoding="utf-8"?>

<ServiceDefinition name="UmbracoCloudService" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition">
  <WebRole name="umbraco452_net35" enableNativeCodeExecution="true">
   <InputEndpoints>
   <InputEndpoint name="HttpIn" protocol="http" port="80" />
   </InputEndpoints>
   <ConfigurationSettings>
   <Setting name="DiagnosticsConnectionString" />
   </ConfigurationSettings>
  </WebRole>
</ServiceDefinition>

The Azure development tools (Fabric etc.) are quite immature in my opinion - very slow to start up (circa one minute) and simply crash when you've done something wrong rather than give a meaningful error message and then exit (for example, when trying to access a local SQL Server Express database (which is wrong - fair enough), the loadbalancer simply crashed with a System.Net.Sockets.SocketException{"An existing connection was forcibly closed by the remote host"}. I have the same criticism of the Azure production system - do a search to see how many people spin their wheels waiting for their roles to deploy with no feedback as to what is going / has gone wrong. Azure badly needs more dev-friendly logging output.

I couldn't get the .NET 4.0 build of Umbraco to work (and it should, .NET 4.0 is now supported on Azure). The problem appears to lie in missing sections in the machine.config file on my Azure machine that I haven't had the time or inclination to dig into yet.

You'll also find that the following directories do not get packaged up into your Azure deployment package by default: xslt, css, scripts, masterpages. To get around this quickly, I just put an empty file in each directory to force their inclusion in the build. If these directories are missing, you will be unable to create content in Umbraco.

Exercises for the reader

* Convert the default InProc session state used by Umbraco to SQLServer mode (otherwise you will have a problem once you scale out beyond one instance on Azure). Starting point is this article - http://blogs.msdn.com/b/sqlazure/archive/2010/08/04/10046103.aspx, but google for errata to the script - the original script supplied does not work out of the box.

* Use an Azure XDrive or similar to store content in one place and cluster Umbraco.

Using Ninject as your Dependency Injection container in ASP.NET MVC 3

2010-08-18T22:36:00.004+01:00

MVC 3 Preview 1 has been available for a few weeks now from Microsoft, with Preview 2 scheduled for release sometime next month.

As a web development framework, MVC 3 is pretty cool - simple to set up and start using, with a terse, clean syntax courtesy of the new Razor view engine. Coupled with Entity Framework 4 (supporting both code-first generation of database schemas and wrapping existing database schemas), MVC 3 + EF 4 has the makings of a very good web development stack.

If you're interested in using Ninject as the Dependency Injection (DI) container in MVC 3, then you'll find the code below interesting - I couldn't find this anywhere else on the web so ended up writing it. It's the required implementation of the System.Web.Mvc.IMvcServiceLocator that gets instantiated and used in the Application_Start method in Global.asax.cs.

Using DI with MVC 3 makes a lot of sense - we use it to decouple concrete implementations from the interface that we code against so that we can quickly swap in alternate implementations, e.g. a quick, self-contained in-memory database for unit testing using Moq or similar.

This link from Brad Wilson shows how to set up Microsoft Unity as the dependency injection container and this presentation from Phil Haack gives a fleeting, tantalising glimpse of how the Ninject equivalent might look but there's nowhere to get the complete code you need to get it working!

So I put the two together in order to use Ninject as my DI container. Here's the code (with zero comments as per my normal coding standard):

using System.Web.Mvc;
using System;
using System.Collections.Generic;
using Ninject;

namespace AdminApp.Models
{

public class NinjectMvcServiceLocator : IMvcServiceLocator
{
public IKernel Kernel { get; private set; }

public NinjectMvcServiceLocator(IKernel kernel)
{
Kernel = kernel;
}

public object GetService(Type serviceType)
{
try
{
return Kernel.Get(serviceType);
}
catch (Ninject.ActivationException e)
{
throw new System.Web.Mvc.ActivationException("PAK", e);
}
}

public IEnumerable<tservice> GetAllInstances<tservice>()
{
try
{
return Kernel.GetAll<tservice>();
}
catch (Ninject.ActivationException e)
{
throw new System.Web.Mvc.ActivationException("PAK", e);
}
}

public IEnumerable<object> GetAllInstances(Type serviceType)
{
try
{
return Kernel.GetAll(serviceType);
}
catch (Ninject.ActivationException e)
{
throw new System.Web.Mvc.ActivationException("PAK", e);
}
}

public TService GetInstance<tservice>()
{
try
{
return Kernel.Get<tservice>();
}
catch (Ninject.ActivationException e)
{
throw new System.Web.Mvc.ActivationException("PAK", e);
}
}

public TService GetInstance<tservice>(string key)
{
try
{
return Kernel.Get<tservice>(key);
}
catch (Ninject.ActivationException e)
{
throw new System.Web.Mvc.ActivationException("PAK", e);
}
}

public object GetInstance(Type serviceType)
{
try
{
return Kernel.Get(serviceType);
}
catch (Ninject.ActivationException e)
{
throw new System.Web.Mvc.ActivationException("PAK", e);
}
}

public object GetInstance(Type serviceType, string key)
{
try
{
return Kernel.Get(serviceType, key);
}
catch (Ninject.ActivationException e)
{
throw new System.Web.Mvc.ActivationException("PAK", e);
}
}

public void Release(object instance)
{
try
{
Kernel.Release(instance);
}
catch (Ninject.ActivationException e)
{
throw new System.Web.Mvc.ActivationException("PAK", e);
}
}

}
}

And here's how to instantiate and use it in Global.asax.cs:

var kernel = new StandardKernel(new NinjectRegistrationModule());
var locator = new NinjectMvcServiceLocator(kernel);
MvcServiceLocator.SetCurrent(locator);

Finally, here's a sample NinjectRegistrationModule which maps the implementation I want onto the generic interface that my code consumes:

using Ninject.Modules;
using AdminApp.Controllers;

namespace AdminApp
{
class NinjectRegistrationModule : NinjectModule
{
public override void Load()
{

Bind<ISpecialRepository>().To<DbSpecialRepository>().InRequestScope();

}
}
}

Effect of the SCEA study guide on the exam

2010-07-23T14:50:00.003+01:00

The SCEA study guide book - especially chapter nine - is already having an effect on the exam. And that effect is interesting, mostly positive but with some negatives as well.

In general, it is fair to say that the overall standard of submissions has improved, and a lot of submissions clearly contain cues from chapter nine of the book - naming conventions, diagram layout, adoption of the server A and B spec approach for the deployment diagram - it's all there in a lot of submissions.

The book has made some of the submissions more anodyne / bland / standardized, which in turn makes me a little sentimental for the past. There's nothing like trying to traverse a crazy class diagram late at night for keeping your brain sharp!

In my opinion, a small but not insignificant percentage of candidates (a bit less than 10%) actually end up submitting a **worse** assignment under the influence of the book, and for a very interesting reason. If you buy the book and read it and aren't an architect, then you will have an incomplete understanding of the concepts covered within it. By extension, when you apply the book material to your submission, there is a very good chance that you will make mistakes that are pretty glaring. So the book will make your submission worse, not better.

As a corollary, if you buy the book and really get the material, your application of that new-found material on top of your already substantial knowledge and skills will result in a strong submission.

In summary then, the book is not a magic book.

The interesting medium / long-term question is whether or not the exam should always have a pass rate of X% and a fail rate of Y% or if it is acceptable to have X approach 100% as a result of the book (that's not happening but clearly it could).

Book - feedback so far

2010-03-27T16:42:00.002Z

The book has just gone back to the printers for a second run. Apparently the first print run (a few thousand I think?) was chewed up by Amazon and direct pre-orders. It's fantastic for that many people to have the book and I really hope it helps you in preparing for the exam.

So, the feedback so far: the reviews on Amazon (both .com and .co.uk) are for the old book, not the new one. Amazon just copied the reviews across (the last one was written two years before the new book published).

So all I've got to go on are comments that I've received directly. Broadly speaking, reviewers fall into two camps:

1. Those who like the ~200 page guide / map to a much larger body of research material (happy);

2. Those who want / expect to find all of the revision material in one book (not so happy).

Our goal was always to write a book that did not replicate the reams of material that exist for the JEE platform. We simply saw no point in doing that. Instead, we wanted to write a book that the candidate could use to:

1. Construct a revision schedule for Part One;

2. Understand how to approach Part Two - constructing your own solution for a given business problem using the JEE platform;

3. Prepare the candidate for Part Three - defending your Part Two submission and explaining how you solution satisfies various NFRs (non-functional requirements).

Broadly speaking, I think we've hit the goals we set. There is an errata list that will be sent to the publisher for the second print and will be published here as well for the purchasers of the first run.

SCEA book publication and shipment dates

2010-01-26T20:53:00.006Z

The book has gone to the printers! It comes off the press on Monday February 1st and gets to Pearson's warehouse on February 4th. From there it usually takes a week to get to Amazon (in the US). Here's the Amazon US and Amazon UK links. It's also available for the Kindle.

People who placed pre-orders for hard copy editions will receive their shipment first - shipped direct from Pearson's warehouse next week.

As far as the online edition goes, the Rough Cut disappears after the final update (which matches the printed book) and it then becomes part of the regular Safari Library and is accessible to all subscribers.

If these dates change, I'll put out another update. It will be fantastic to see the book finally out there!

Book - chapter nine available for download

2009-11-29T21:52:00.002Z

I've put a PDF copy of chapter nine up on www.box.net for download here. Chapter nine is two things - a seminal chapter in terms of the exam content it covers (Parts II and III of the SCEA exam) and also one that Safari Rough Cuts (SRC) keep missing out on in updating it. The version of the book on SRC is a lot older than this version. I'll keep this download link live until SRC is updated with the latest version. Enjoy.

Google Go overview for JEE and .NET architects / developers

2009-11-26T17:45:00.005Z

A few days ago, Google released a pre-release / early look copy of a new programming language and platform - Google Go - for general consumption and feedback. After working with Go for a couple of days, I took my notes and thoughts on how it compares to the Java (J2EE / JEE) and .NET platforms - specifically as far as enterprise computing goes - and wrote them up into this article. All feedback appreciated.

From the Conclusion: "Go has the potential to completely supplant both C and C++ in the systems programming space and that is clearly it's first goal, but it also has the potential to do much more than that - to reach into the enterprise computing / web+application tier as well. It would be short-sighted in this day and age to design a platform that was restricted to low-level systems programming only, when most companies invest far more time and effort in building applications that are used by consumers to communicate, collaborate, transact ecommerce and more. In this author's opinion, Go has the legs to go further".

Another book update (aka where is it..)

2009-11-26T01:42:00.004Z

The book is still moving through pre-production at Pearson. I'm sure a good number of you will be wondering what kind of impetus can be administered to push it along and I feel the same way myself!

It looks like the book content got caught up in a major upgrade to Safari Rough Cuts that is just about sorted now. A new version has just published, but I know that it does not have the latest chapter nine content that I wrote last month. Chapter nine is a key chapter in the book, focusing as it does on the JustBuildIt scenario that I outlined at JavaOne 2009 and solves it completely. This chapter maps onto Parts II and III of the exam, where candidates download a business scenario of equal complexity and have to propose and defend a solution to it.

I'm waiting to see when this chapter will show up on Rough Cuts. If it's only a few days away, I'll write another post advising that it's there. But if the delay is going to be as long as the previous delay (essentially four - six weeks to get some content updated), then I'll publish chapter nine as a PDF file on www.box.net, like the other sample chapter and the JavaOne 2008 and 2009 slides. Again, apologies for the delay - I'm pushing as hard as I can.

Addressing book comments on previous post

2009-10-22T14:29:00.003+01:00

I just wanted to address some of the comments posted against the previous post in a standalone post (posting my comments against previous comments inline isn't a great way for people to see replies).

A new version of the book was pushed to Rough Cuts late last week and there is another one waiting to go as well. As I understand it from Pearson, Safari are in the middle of a software upgrade at the mo, hence the tardiness in pushing updated content onto Rough Cuts.

Once, the next version of the book makes it onto Rough Cuts, I'll be happy that it represents the content for most of the chapters that you'll see in the published book, i.e. go ahead and use it to revise. The one exception is chapter nine, where I consider the content in that chapter to be the bare minimum you need to provide in part two of the exam to pass. We'll be beefing up the UML diagrams and associated English text in that chapter. So the content in chapter nine is correct, but not yet sufficient, for part two.

SCEA study guide - progress update

2009-10-15T08:46:00.003+01:00

We're pretty much finished with the book edits now. One chapter still looks light and that content will be finished this weekend. But we've underestimated the time the publication process takes at Pearson. Greg tells me that we're looking at late December / early January for the book to be physically on the bookshelves.

Apologies for setting expectations of mid-October in my blog posts - I didn't realise the time the publication process takes. To mitigate this publication delay, we'll be getting the Rough Cuts content updated with the latest version as soon as possible.

Addressing Rough Cuts feedback received

2009-09-12T14:45:00.003+01:00

Since the book draft has been released on Safari Rough Cuts, I've received feedback on it, both as blog comments and also directly through the Rough Cuts feedback mechanism.

Focusing on the areas of improvement that people have commented on:

1. The final version of the book is significantly longer than the rough cuts version, but it is no weighty tome either. The whole point of the book is to focus on the material that we believe is directly relevant to the JEE Architect exam, as opposed to the JEE platform itself;

2. The next version of the book contains chapter six (JEE security) which is missing from the current Rough Cuts version - that was a trade-off that we took - to get the book on Rough Cuts in order to get reader feedback, without chapter six; and

3. The next version of the book contains a completed chapter nine - a fully-worked Part Two and Three solution to an assignment (the JustBuildIt business problem) of similar complexity to the exam assignments.

Thanks for all the feedback so far - it helps us to make the book better. Unfortunately, time does not permit me to respond to feedback left anonymously, but rest assured that your comments have been read. Finally, we're still shooting for the October date to production.

Book update

2009-08-30T19:10:00.002+01:00

Well, the book edits continue apace, and most of the artwork has been sent to the Pearson graphics team to be transformed from the raw, blocky diagrams that we have created into beautiful images for the book.

We're basically left with chapter six and nine edits now. There's a bank holiday tomorrow so I'm planning on using that to get my work done and then send it to the editors for review. We're almost there!

Humphrey

SCEA study guide now on Rough Cuts

2009-07-01T22:22:00.001+01:00

Well, the book draft has finally made it onto Rough Cuts here. All feedback is truly appreciated and we'll be looking to get a more rounded / completed version up as soon as possible.

Sample SCEA book chapter for download - web tier

2009-06-21T16:18:00.002+01:00

In anticipation of the full book going up on Rough Cuts any day now, I wanted to share one of the chapters to get early feedback on it. The chapter I'm sharing is the third chapter focusing on the web tier portion of the exam and is still in draft status, but with the exception of the graphics needed, is more complete than not. All feedback appreciated. Link to chapter.

SCEA study guide book update

2009-06-19T13:52:00.002+01:00

Just a short update on the book for all of the comments asking for an update. I put the content on the Pearson FTP server for peer review and editing the Saturday before JavaOne. Unfortunately it's taking longer than expected to get it pushed through that review process - I'm asking pretty regularly and I'll keep asking on your behalf. As soon as it goes up on Rough Cuts I'll let you guys know.. thanks for the continued interest in the book!

Slides from JavaOne BOF on the SCEA exam

2009-06-11T17:20:00.001+01:00

I've posted the slides for my BOF talk at JavaOne - enjoy! (including the obvious typo on slide 16).

Sample assignment for the SCEA exam - part two

2009-06-11T17:11:00.001+01:00

So, as a follow-up to part one of this little series, below is the exact list of deliverables that you are required to develop for part two of the exam.

It is your task to create an architecture and design for the System under Discussion (SuD) with the given business domain model, information provided above and requirements in the use cases. The architecture must be built using the JEE platform. All deliverables will be accepted as HTML only and each diagram must be UML compliant.

1.Create a class diagram for the SuD. Public method names referenced in other UML diagrams (e.g. sequence diagrams) should be provided.

2.Create a Component diagram for the SuD showing the components used in the system and their interaction. Examples of components are EJBs, Servlets, JSPs, major POJOs (Plain Old Java Objects) and important Managers / Controllers / Design Pattern implementations.

3.Create a Deployment diagram that describes the proposed physical layout of the major tiers of the SuD.

4.Create either a Sequence or Collaboration diagram for each use case provided.

In addition to these UML deliverables, the exam requires you to:

1.List the top three risks and identify a mitigation strategy for each risk.

2.List any assumptions made during the process of coming up with the architecture and design.

Your architecture and design will be graded on how well it supports the requirements detailed in this document and on the clarity of all information provided in both textual and diagrammatic form.

Sample assignment for the SCEA exam - part one

2009-06-09T11:54:00.003+01:00

Following up on last week's BOF at JavaOne, I promised to put the slides online as well as the expanded version of a sample scenario for part two of the exam. The slides are coming later, and here is the expanded version of the JustBuildIt scenario. As per the talk, the goal here is to expand on this scenario over time and create a fully-worked solution from it. In subsequent posts, I'll be posting the domain model, deliverables and use case diagrams so that you can see the full complexity of a scenario that I believe is of the same complexity as an actual scenario in the architect's exam today.

Enjoy!

You are the architect for JustBuildIt Corporation, an international, vertically-integrated construction company with significant operations in the US and Canada, Europe and the Pacific Rim. JustBuildIt operates its own forests, quarries and steel foundries to supply its own building sites with wood, concrete and steel. This end to end style of operation has helped JustBuildIt to keep its costs of raw material down in an era of soaring commodity prices. The management team has recently concluded a business-wide review from leaves to roots of the entire company and one fact is apparent – JustBuildIt pays a lot of money moving raw materials to construction sites, even when there are materials just as suitable nearby.

JustBuildIt has decided to build a building commodities exchange to allow it and some of its competitors to pool excess capacity in a co-opetition model. In the future, raw materials for a construction site will be sourced through the exchange, rather than exclusively from JustBuildIt inventory.

Based on the management’s report and also interviews with key senior staff, you know the following:

• JustBuildIt have recently invested in an inventory and order management system which tracks capacity of their production facilities and also individual orders coming in from construction sites around the world. This system is accessed via a JMS Queue;

• JustBuildIt have decided to expose the interface to their exchange as a web services API;

• In order to counter accusations of unfairness, JustBuildIt has agreed with all participants that 95% of all transactions to and from the exchange will execute in 5 seconds or less, with the remaining 5% executing in 10 seconds or less;

• The system has an uptime requirement during core working hours (GMT -8 to GMT +8) of 99.99%; and

• The actual placement of orders into the exchange is a manual process – JustBuildIt site foremen place orders daily based on individual construction site requirements.

Short book update

2009-04-26T22:26:00.003+01:00

Just a quick update on the book. As with all things literary, this book has taken on a life of its own, and Mark and I missed our self-set date of end Feb to have the book with Pearsons for final review and publishing. But the good news is that we're close to having a draft copy up on Safari - i.e. we're shooting to have the book up on Safari in the next two weeks. I'll post the link as soon as I have it.

Presenting at JavaOne 2009 - SCEA Take Two!

2009-03-07T20:45:00.003Z

Sun have accepted my BOF proposal for JavaOne 2009, so it's time to start looking for flights to SFO again and making sure that the hotel I'm staying at isn't right in the middle of the Tenderloin..

My proposal this year built on the material covered in last year's BOF, and I'll also take time to dig deeper into Part Two of the exam (where a lot of people come a cropper) and work through exactly what the examiners are / aren't looking for and why.

If you're going to JavaOne in this crunch year, looking forward to seeing you at the BOF!

Skillsmatter - Oct 16th presentation on SCEA

2008-09-06T21:44:00.003+01:00

Sun UK have kindly invited me to speak as part of the Skills Matter program in London on October 16th. I'm looking forward to it and I'll be covering the topic of the Sun Certified Architect Exam (SCEA) for JEE 5 at this topic. A great way to learn about the exam if you're in London and are interested in pursuing this certification.

More details on the talk here.

Slides from BOF 5205 at JavaOne 2008 now available

2008-07-20T19:44:00.004+01:00

Following a number of requests, I've put the slides from BOF 5205 (All About the Sun Certified Enterprise Architect (SCEA) Exam) online here. The BOF was designed to give attendees an end to end overview on the exam as described below.

The Sun Certified Enterprise Architect (SCEA) exam represents the
culmination of Sun's certification program for the Java™ community. This
session covers the newly revised SCEA certification that is due to be
released in the first quarter of calendar year 2008. In this BOF, attendees will learn about the three parts of the exam and what they test. They will also learn the following:

• Who is the target audience for the exam

• Which exam rumors are true and which are false

• Tips for exam preparation, including preparation materials

• Tips for taking each part of the exam

We'll cover how the exam has been completely rewritten to cover the Java EE 5 platform, the design decisions that went into the new exam and the rationale and motivation behind them (and how this affects candidates).

Perennial exam rumors will be detailed and addressed, we'll dig through the mountain of potential revision material to single out the most relevant books, specifications and articles to focus on and we'll provide practical tips on how best to approach parts one, two and three of the exam.

If you are interested in becoming a Sun certified enterprise architect, this BOF is definitely for you. If you haven't considered architect certification before, stop by anyway - you'll learn what becoming a Sun Java EE 5 architect can bring you and how effective the exam preparation is in rounding you out as a Java EE 5 architect.

JavaOne 2008 BOF proposal accepted

2008-02-24T10:36:00.007Z

See you at JavaOne this year if you're going! I'll be co-presenting a BOF with Mark Cade on the new revised JEE 5 Architect exam from Sun Microsystems. Mark and I participated in the workshop to create the multiple choice questions and we designed and implemented parts two and three, plus right now we're heads down writing the book to go with it (phew!).

We feel that there is a lot of latent interest in the exam and we wanted to give a clear presentation on what taking the exam / certification entails and give guidance on some common gotchas that we see all the time in submissions. Finally we wanted to dispel common misconceptions that circulate perennially (my written english won't be good enough etc.). Check out the BOF description here.

Part two of the multi-core computing series

2006-11-28T16:21:00.000Z

Check out part two of my Javaworld series on multi-core computing here.

This article builds on Part One and describes and implements a simple threaded framework leveraging Java 5/6 features that you can use to build benchmarks pertinent to your application as you look to find the best hardware for your system. The source code for the article is here. Enjoy!

Technorati post

2006-09-02T13:00:00.000+01:00

Technorati Profile

New Javaworld article on multi-core computing

2006-08-03T17:16:00.000+01:00

My latest Javaworld article details the issues around multi-core computing from a Java perspective with some initial discussion on the theory behind parallel computing (Amdahl, Gustafson et al.).. From the article "Hardware is really just software crystallized early," says Alan C. Kay in his paper "The Early History of Smalltalk" (ACM, 1993). This quote really explains the inspiration for this article. Software developers have always been at the mercy of hardware manufacturers, although we've had a pretty easy ride of it since the inception of the computing industry itself. From then until now, increasing speeds of every single component that goes into the standard Von Neumann architecture have given our software literally free increases in performance.

No longer."

This is a good article to read in order to understand how all the major vendors are in the middle of a concerted move to multi-core computing, how that affects the software you design and develop and Java support for multi-core computing.

Check it out here

Can't we just keep it Simple? (Javaworld SOA article)

2006-01-10T15:43:00.000Z

Check out my latest JEE / SOA article on Javaworld here.

The core premise of the article is the KISS (Keep it Simple, Stupid!) principle - SOA has definite benefits when applied to some JEE projects, but not to all and developers need to remain wary of "silver bullet" promises made by SOA vendors.

Dearth of production-ready JMX Consoles

2005-09-20T17:25:00.000+01:00

It turns out that although JMX is a mature server-side specification addressing how to remotely monitor and manage a J2EE application, it leaves a lot to be desired in terms of the presentation of that information in a UI - whether it be thin or thick client (HTML or Swing in the Java world).

Various implementations of JMX UIs then leave quite a bit to be desired - system administrators don't manage systems using an MBean concept, they use processes. In addition, I believe that MBeans should be nestable - so that I retrieve a top-level MBean corresponding to my application and then use it to drill-down to MBeans corresponding to specific modules in my application that I want to manage as part of a sys admin process.

I've chosen to go with a regular thin client suported on IE / Firefox for this - I simply don't need Swing and requiring my end-users to install a heavy-weight client such as MC4J plus potentially a JRE just to manage my application is a requirement too far..

Java Management Extensions - critical for Enterprise Applications

2004-12-15T17:31:00.000Z

Java Management Extensions or JMX for short have been around for a long time now (in technology time at least). I first came across JMX at JavaOne 2000, where a guy from Sun Microsystems France went through what it could do. It didn't take off then - although BEA and JBoss both added JMX to their servers. In the case of JBoss, JMX was crucial to the micro-kernel architecture and still plays a large part today - although they seem to be much more enamoured with AOP now! ;-)

I'm using mx4j and mc4j on a current project to really beef up the remote administration capabilities for one of my projects. It's working out well, although the lack of documentation around mc4j is an issue. I chose mc4j because I found the default HTML view from mx4j to be not enterprise-class and I like the dashboard feature in mc4j. I certainly don't like the netbeans dependencies though.. extending mc4j is needlessly complicated from a compile, build, test point of view..

Anyhow, I'm now at a point where I'm exposing many hitherto locked-down features of an application easily via the mc4j client connecting to the mx4j runtime as a JSR160 compliant application and it's working out well - next to get some dynamic charts working using jfreechart!

And moving back away from Linux as a primary desktop OS!

2004-12-15T17:30:00.000Z

I figured an update on my earlier post was warranted, given my failure to move away from Microsoft Windows.

My successes below still stand, but there were also some unanticipated problems. These problems included:

1. Annoying differences between the Eclipse IDE on Windows and on Linux (it is way more usable on Windows, esp. keyboard shortcuts);

2. Microsoft lock-in alive and well - I have to take my pst files with me, but they just didn't want to come. More investigation needed here, also I couldn't use Evolution or the Exchange connector because of server-side issues; and

3. Speed - subjective opinion, but I felt that the speed-up I gained in not needing Sophos or Symantec running was cancelled out by the lardiness of Gnome / KDE. Yes I'm aware of faster WMs out there, but they don't look as nice or have the same level of features.

I ran into additional issues where complex formatting in Microsoft Word documents didn't render properly in OpenOffice, problems accessing networked printers etc. - no showstoppers, but not something that I want to wrestle incessantly with either. So I'm still in dual-boot land, planning to move away from Fedora 2/3 in the near future and over to SuSE - I've read good things about it especially when used in a corporate environment. Hopefully it's more laptop-friendly too.

Moving towards Linux as a primary desktop OS

2004-12-15T17:29:00.000Z

Over the past few days, I've been doing a lot of work getting ready to move to RH Fedora Core 2 as my primary desktop OS. I've had linux distros of one form or another (but usually RH) on my machines since 1997 but I've never been comfortable relying on it for my day to day job. Simply put, I need a desktop that works.

So I've chosen RH Fedora Core 2. I would have gone with SuSE 9.1 and Evolution but I spent a weekend trying to get the bloody ftp installation to work and failed miserably..
Successes so far include getting my previously linux-averse WiFI card (US Robotics 2210) working via the acx100 driver (thank you folks!), moving over my J2EE servers of choice and getting comfortable with the new keyboard shortcuts for Gnome and firefox.

Major tasks that still await me are moving from outlook to thunderbird (including migrating a 600 MB pst file), getting APM/ACPI working properly and various other linux tweaks for the thinkpad t30.

Oh, I've finally been using up2date and yum in anger and once configured properly they save loads of time over manually managing rpm dependencies.

Towards a Java Research Map

2004-12-15T17:28:00.000Z

Recently I posted a blog entry to Javaworld covering my initial attempt to map out the world of Java as I see it with a view to setting some direction for my R&D efforts over the next few months. The diagram has some interesting properties - check out the javaworld blog entry for more details.

In Pursuit of Perfection

2004-12-15T17:25:00.000Z

Please use this entry to record your comments on my latest Javaworld article titled In Pursuit of Perfection. The article is here - http://www.javaworld.com/javaworld/jw-01-2004/jw-0102-perfect.html

Looking back on this article now, I think I tried to cover too much - a correct and indepth treatise of the perfect technology platform would probably need a book to cover properly.. well I got a few good flames for my "lightweight" article, notably from Daniel on www.java.net!

The Awards are Out!

2004-12-15T17:24:00.001Z

Check out the 2003 JavaWorld awards here - http://www.javaworld.com/javaworld/jw-06-2003/jw-0609-eca.html

All of the category winners are worthy victors, although I have to say that I was very disappointed that the eclipse IDE (www.eclipse.org) didn't win anything - it has had a profound effect on how I code and I think it's a cracking good IDE. The eclipse AOP implementation AspectJ did win an award however - Most Innovative Java Product or Technology. This is the equivalent of the Best Picture Oscar as far as the JavaWorld awards go..

Well done to all concerned - it was tough being a judge and having to choose between so many strong offerings (no it wasn't I'm just saying that ;-p)

Keeping up to date in Java

2004-12-15T17:24:00.000Z

It's that time of year again - the annual Java love-in in San Francisco. Due to work commitments, I can't get to Java One this year, although to be honest, I'm looking for a more advanced conference anyway - a lot of the JavaOne sessions are targeted as "101" sessions. Still, there's already been a flurry of announcements that I want to digest to see if they affect our product architecture moving forward. Some look exciting - more on that later.

Keeping up to speed with JavaOne also raises a wider issue - how to keep up to speed with the Java technology platform in general. With Sun looking like they are going to really up the ante on new additions to the J2SE, J2EE and J2ME platforms, it's going to get more and more time consuming.

Push technologies - e.g. RSS feeds for blogs and key web sites are a great help. Good blogs are hard to find though, and I tend to use blogs to generate new ideas or areas to explore - I don't buy opinions that are expressed in a blog without doing my own R&D first - the writing style tends to be polemic in order to attract an audience / get a reaction as opposed to even-handed and impartial.

Feedback on the J2EE Architect Certification article

2004-12-15T17:23:00.000Z

[Apologies to those people who added comments to this post on my radio blog - they didn't come over..]

Well, the article on the SCEA J2EE Architect certification has garnered a respectable level of feedback from the J2EE community - both direct feedback via emails and also through the discussion thread on theserverside.

On the discussion thread, a number of common themes have come up and I'll address the main one in my opinion - the validity of the certification.

Some posters have recounted stories about companies where one person sits the exam and then discloses exam details to colleagues, effectively giving them a free ride. Other stories include the 3-hour submission and so on.

Long story short, I addressed this type of thing in the article. It sucks to see people cheat because they are attempting to devalue all the hard work that decent candidates put into getting the J2EE architect certification, but the fact is that some people will always try to cheat on an exam.

Cheaters will always be found out when their experience and know-how falls woefully short of their resumes / CVs and any good interviewer will quickly figure this out. Plus they won't take 1% of the value that comes with actually working through the exam itself. I have yet to hear of anyone who walked into a position - contract or permanent - simply because they had the J2EE architect certification. Especially with the job market as tight as it is right now on both sides of the atlantic, that just ain't gonna happen.

As for people who boast of a successful submission in some ridiculously short timespan - they cannot prove it. Having taken the exam and also seen submissions, I don't believe that a successful submission could be produced in a couple of hours. If someone wants to boast about speed of submission *shrug* so what? Let me know what you think - add a comment!

J2EE Architect Exam

2004-12-15T17:21:00.000Z

This article covers the SCEA J2EE Architect exam. theserverside.com picked up on the article and that garnered some robust discussion on the exam and exactly how valuable it is. Here's the article - http://www.javaworld.com/javaworld/jw-12-2002/jw-1206-architect.html

JBoss Review

2004-12-15T17:18:00.000Z

Here's a review I did on JBoss four years ago now (December 2000! doesn't seem that long...) There have been a couple of follow-on reviews since this one at theserverside - I liked David Jones' the best. It's cool to see how far JBoss has come. At the time, I had to ask Floyd Marinescu to create a section for JBoss - up until then he had regarded JBoss as not really an app server! I was a judge at the Javaworld awards this year at Java One and JBoss scooped Best Application Server award, so their star is most definitely in the ascendancy at the moment. And what with JBoss 4 and all this hoopla about AOP, they look to be still innovating.

As long as BEA and IBM insist on charging high licensing fees per CPU, JBoss will continue to wipe the floor with them in terms of market share. Interesting to see the Business Process Modelling (BPM) moves as well - I'm happy to see JBoss make money, but now when I see new technology from the group, I have to wonder is it useful technology or technology that begets maintenance and services revenue.

http://www2.theserverside.com/reviews/thread.jsp?thread_id=2918&;article_count=47

J2EE versus .NET

2004-12-15T17:15:00.000Z

Update: since I wrote this article, my opinion of .NET has changed, matured even. In my opinion, Microsoft have added significant desktop integration and rich client features to the .NET platform that other technologies need to address and quickly.

This article took a lot longer to write than I anticipated. Don't tell Microsoft though, or they'll probably publish this quote claiming that J2EE articles take longer to write than .NET equivalents :-) Michael Monteiro and I pooled forces to put together a balanced assessment of J2EE versus .NET. You may find our recommendations bland and "on the fencey", but they're accurate - there is no "winner". J2EE stands for a Java-based cross-platform technology stack, .NET stands for a multiple-language based single platform technology stack.

Interestingly, a lot of the reader feedback centered on exactly how truthful Microsoft is in claiming that .NET is truly viable for multi-language development. Other readers were quick to point out the hidden effort in porting existing applications to the .NET platform. Javaworld were kind enough to have a special follow-on issue of reader's letters to capture this feedback -

http://www.javaworld.com/javaworld/jw-07-2002/jw-0726-letters.html

Bottom line #1: J2EE and .NET are both viable systems for enterprise development.

Bottom line #2: I like .NET. I'm also happy to use Windows as well as {U | Li} nix. Come to think of it, I don't particularly have a problem with any Microsoft technology (except for the security issues). It's the company I have a problem with. I also think that the only entity powerful enough to rein in Microsoft will be Microsoft itself - through exorbitant licensing practices that erode it's own market share.

http://www.javaworld.com/javaworld/jw-06-2002/jw-0628-j2eevsnet.html

To EJB or not to EJB

2004-12-15T17:13:00.000Z

The background to the next article is pretty interesting - it was created solely because of reader demand! Take a look at "top ten J2EE dangers" danger # 2. Look familiar? Yup, it's the old to EJB or not to EJB question.

Check it out: http://www.javaworld.com/javaworld/jw-12-2001/jw-1207-yesnoejb.html

I'm especially proud of my Hamlet take-off, but then again when I was in school (high school for you state-siders) my art teacher used to hold up my paintings so that the class could have a laugh, so my creative opinions are pretty suspect... :-)

This article is even more pertinent today - large swathes of the latest J2EE specifications, in particular EJB 3.0 are all about reducing complexity - even to the extent of auto-generating a lot of the code that goes to make up an EJB.

Top Ten J2EE Project Dangers

2004-12-15T17:08:00.000Z

This article is a perennial evergreen - as evidenced by the hits it continually generates on javaworld.com and the fact that it accounts a large proportion of the emails I get on my articles.

The article is http://www.javaworld.com/javaworld/jw-03-2001/jw-0330-ten.html

and I would classify it as an anti-pattern article. It lists out the top ten reasons in my opinion responsible for J2EE project failures. Read it and weep - we've all been here at some point in the past!

Frameworks save the day

2004-12-13T09:26:00.000Z

This article sprang directly from the fact that I found most of the commercial and open-source offerings too complicated or unstable, so I built my own lightweight and easy to use framework to handle the bread and butter tasks in Java / J2EE development:

http://www.javaworld.com/javaworld/jw-09-2000/jw-0929-ejbframe.html

This article was very well-received. Based on the emails I received, I reckon ~ 200 companies are using it world-wide, including at least three Fortune 500 companies. This article is 4 years old now (an age in Java development time) and I still get emails on it - so reducing complexity is just as hot a topic now (2004) as it was back then..

First blogspot post

2004-12-08T12:27:00.000Z

test post - wonder if google is more "efficient" at indexing blogger.com than radio? :-)