There is a rather interesting event in Denver this week called the Defrag 2008 Conference.  Check out the agenda - much discussion on content, information, flow, computing, applications, etc.  I received several commentaries over twitter from Ian Glazer, senior analyst with the Burton Group.  One I found particularly poignant.

We are in an asymmetric conflict with information, fighting a modern fight with cavalry and sword.

Profound? Yes. Obvious? When we consider it. Actionable? I'll let you know when I'm caught up on email, blogs, magazines, books, whitepapers...

=rob.ciampa

The Rob Daly piece from Waters that I referenced last week really got me thinking.  I responded to the piece in Waters and will reprint it below, but the topic has been a source of ongoing debate. Recently, I was out with several customers where we spent dinner discussing the physics of information movement within complex systems.  The gist was that we continually tweak our market data systems to ensure low latency, yet we just don't have the time (or the insight) to look at it more systematically.  This is a problem.

The Waters Letter

Rob Daly's September column, "Stop the Latency Insanity," is a harbinger for a much broader discussion on a systematic approach to addressing and analyzing latency across the entire trade-execution spectrum. The ever-elusive latency equation is inherently complex and consists of diverse variables whose individual characteristics are constantly changing. Mr. Daly covers several of these, including network bandwidth, protocol stack bypass, application architecture, and so forth, but we need to look at the entire chain, not just the pieces.

Unfortunately, we struggle as an industry to fully grasp the entire latency spectrum. Why? The reasons are as diverse as the variables. Trade execution spans departmental and organizational boundaries, preventing integrated and holistic analysis; latency tools only cover a limited number of discrete systems; volatility and the non-deterministic nature of market data inhibit accurate modeling; algorithmic execution generates non-deterministic computational and network loads.

The semblance to chaos theory is warranted. We're dealing with complex, inter-related, dynamic systems, which is why latency is so hard and-stealing from Mr. Daly's point-insane. While chaos theory puts much emphasis on initial conditions (market open), today's trading ecosystems introduce more stimuli all the time (Fed announcements, earnings reports). The mathematical modeling of latency around chaos theory may be a good concept (and a reasonable PhD dissertation), but it's just not pragmatic in the real world.

That shouldn't, however, stop us from attempting to solve the latency equation-which also needs to include stability, predictability and scale. There isn't a choice, especially in today's markets. Not having a grasp on broader, systemic latency will not only put our trading systems at risk, but erase any market advantage as well.

=rob.ciampa

=rob.ciampa

But please, people, get a grip-there is only so much that physics can do with most firms' existing architecture...While reducing latency is an ever-present concern for firms, it must beaddressed in a rational manner, balancing the potential return with itspotential costs.

I agree and I disagree.  Perhaps that's an indication of insanity.  Perhaps there are more ways to look at the latency problem.  Perhaps we'll continue this discussion.

=rob.ciampa

I really like industry events: the intensity; the competitiveness; the pithy debates; the BS; the incredible concentration of opportunity. The good ones have the pulse and cacophony of a third-world bazaar. The evening parties aren't too bad either, especially when they're in New York or Las Vegas.

Fortunately, we had High Performance on Wall Street 2008 in New York this week. It was at the Roosevelt Hotel, La Grande Dame from La Belle poque. I felt out of place without my frock suit and top hat, but mentally returned to the 21^st century and enjoyed the healthy debates on in-line risk calculations, low latency, complex event processing, co-lo architecture, etc.

Candidly, with all the market animation this week, it was tough to gauge what the atmosphere would be going in to the event. With rare exceptions ('Hey, where's the Lehman dude on the panel?') things went rather smoothly. Hats off to Pete Harris for keeping things lively and moving.

I was able to catch some very good panel discussions as well. I'm always leery of panels because well many (regardless of the event) are truly awful. Often panelists are inarticulate, rambling or just too timid to have an opinion. And moderators? They're running the show but often forget their role and responsibility to us not seated on the dais. Far too typically moderators go through the panelists in sequential order asking softball questions. For most of the audience this proves (often after the ubiquitous high-carb lunch) a quick route to slumberland. When I'm in the audience, I want to be entertained and mentally challenged. I want Jerry Springer meets Harvard. Hey Mr./Ms. Moderator: pick a fight, force a debate, enlighten me and stop wasting my time.

Off my soapbox and back to the panels (because this was not much of an issue at this week's event). I very much enjoyed the session 'Gaining First Mover Advantage with a Low Latency Market Data Solution.' I skipped the concurrent 'Low Latency Market Data Distribution Software and Hardware Approaches' panel because I work with Barry Thompson and get to eat, drink and debate with him all the time. What struck me about the first panel was a very poignant comment made by an excellent panelist. He commented on the market obsession with low latency and the collective failure to account for devastating effect of outliers late, non-low-latency information. Algos just don't like outliers and it screws many of them up. Outliers are low-latency impurity. 99.44% pure is good for Ivory Soap, but it's not good enough for today's environments.

This subject deserves much more scrutiny. Outliers undermine low latency. Maybe a debate on this at the Roosevelt next year?

=rob.ciampa

Amazing how time changes everything. My memories were stirred by a recent article in Network World by Steve Taylor and Jim Metzler 'Why it's time to let the OSI model die.' I think it's been six feet under for some time now, but I still think Steve and Jim made their new 3-layer model too complex. Perhaps it's only 2 layers: application and network. We know company XYZ has an application group and a network group, don't we? And, like two siblings, those two groups always get along, don't they? Maybe we do need a seven layer model again, somewhat like The Waltons of technology. And they may get along better, too. I'm imagining my new world 'Rob, I think you'll need to speak with my dear friend in the layer 6 group. I think he's drinking espresso in the cafeteria with the layer 1 and 3 guys.'

OSI. It's dead. Go bury it on Walton's Mountain. So, where's middleware? Layer 2.5. At least it's stillbetter than 3.

G'night John-Boy.

=rob.ciampa

So, what was the motivation for this panel entitled Ultra-Low Latency Messaging Panel Discussion? Events, messages and low-latency communications are critical components of what we do. Roy commented that events are becoming more widely deployed in diverse business settings. More importantly, those events have increasing time sensitivity. Messaging, of course, is the foundation for event processing now and moving forward.

So, what was my take? Aside from the typical 'some speakers were good and some were less so,' one thing was clear: the world of messaging is splitting into software and hardware. To say one is better than the other would be nave and we don't even position ourselves that way. But one thing we do know: when the volumes start getting high, software systems just can't keep up. (When was the last time you used a software-based LAN switch?) Is this new? No, but with the rise of events and their underlying messages, we're starting to redline the performance tachometer.

Where is this all going? Just look to the financial services industry; it's been a rather good harbinger for the broader enterprise.

=rob.ciampa

'The LSE is down,' said the voice on the line.
'You're kidding me,' I responded. 'Please not today. The Fed announced the Fannie Mae and Freddie Mac rescue. Great day to be in the market.'
'Exactly. Lots of volatility.'
'Any idea of the cause?'
'Nope. It's been called a 'connectivity failure' by the LSE.'

Lots of questions around this outage. I'm sure the LSE is not too happy either.

From the Wall Street Journal:

The connectivity failure by the London Stock Exchange Group PLC, the exchange operator, is an "absolute nightmare," said one trader. "We are just not trading on what should have been a big trading day."

Was it an application failure?
Was it a server crash?
Was it a network failure?
Was it a middleware issue?

We'll get more information over the coming days, but rest assured we're seeing the results of building complex trading ecosystems with interdependencies that have their own, often significant risk surface areas.  It's not just an LSE issue.

=rob.ciampa

Some time ago (in the pre-steroids era), we had message traffic flowing on a separate network. It was the right thing to do. Later, that network was collapsed into the 'new and improved' high performance switched infrastructure. All the data lived happily together for a while until message volumes went up and multicast trashed the network. And everyone screamed about low latency. Now we're seeing the pendulum swing back to separate networks for messaging again. We don't have time to manage one network and soon we'll have two.

=rob.ciampa

=rob.ciampa

Why is this not akin to a John Kerry flip flop?

Most recent noise on the The Street had said vendor saying no, no, no to hardware. We, of course, argue that it's inevitable. Look at what the networking world taught us.

Should this legacy messaging be hardware accelerated?

It's not about messaging; it's about a messaging architecture that supports acceleration. More lessons here, this time from the automobile world. Shelby SuperCars introduced a 1,183 hp rocket last year that hit 257.41 mph on the road. It wasn't just about the engine, though. The rest of the car was architecturally matched to it. Putting that engine in the classic Toyota Camry could produce much different and possibly dangerous - results. You be the judge.

Will this make other problems go away, too?

Again, it's about architecture. Nothing like a slow messaging consumer to bring down a peer-to-peer framework even one that's hardware accelerated. You need a new architecture to fix this. Period.

What about all this XML stuff?

I'll answer with a question: How prevalent is XML in the high-performance world? Before anyone barks I'm an XML fan but for the right applications.

Welcome to the high performance world of hardware. May your engine bolts be fastened properly and your drive train sufficiently matched.

=rob.ciampa

=rob.ciampa

=rob.ciampa