Skip navigation
Check out our Message Medium blog to find out what’s going on in the hardware-accelerated, low-latency messaging world. Information ConflictNovember 4, 2008 at 8:00 am
Profound? Yes. Obvious? When we consider it. Actionable? I'll let you know when I'm caught up on email, blogs, magazines, books, whitepapers...
|
There is a rather interesting event in Denver this week called the The Rob Daly piece from Waters that I referenced last week really got me thinking. I responded to the piece in Waters and will reprint it below, but the topic has been a source of ongoing debate. Recently, I was out with several customers where we spent dinner discussing the physics of information movement within complex systems. The gist was that we continually tweak our market data systems to ensure low latency, yet we just don't have the time (or the insight) to look at it more systematically. This is a problem.
Rob Daly's September column, "Stop the Latency Insanity," is a harbinger for a much broader discussion on a systematic approach to addressing and analyzing latency across the entire trade-execution spectrum. The ever-elusive latency equation is inherently complex and consists of diverse variables whose individual characteristics are constantly changing. Mr. Daly covers several of these, including network bandwidth, protocol stack bypass, application architecture, and so forth, but we need to look at the entire chain, not just the pieces.
Unfortunately, we struggle as an industry to fully grasp the entire latency spectrum. Why? The reasons are as diverse as the variables. Trade execution spans departmental and organizational boundaries, preventing integrated and holistic analysis; latency tools only cover a limited number of discrete systems; volatility and the non-deterministic nature of market data inhibit accurate modeling; algorithmic execution generates non-deterministic computational and network loads.
The semblance to chaos theory is warranted. We're dealing with complex, inter-related, dynamic systems, which is why latency is so hard and-stealing from Mr. Daly's point-insane. While chaos theory puts much emphasis on initial conditions (market open), today's trading ecosystems introduce more stimuli all the time (Fed announcements, earnings reports). The mathematical modeling of latency around chaos theory may be a good concept (and a reasonable PhD dissertation), but it's just not pragmatic in the real world.
That shouldn't, however, stop us from attempting to solve the latency equation-which also needs to include stability, predictability and scale. There isn't a choice, especially in today's markets. Not having a grasp on broader, systemic latency will not only put our trading systems at risk, but erase any market advantage as well.
The Scariest Halloween Yet
October 31, 2008 at 3:00 pm
What a way to end the month of October 2008. Let's look at the VIX, Chicago Board Options Exchange Volatility Index.
Low Latency (In)sanity?
October 30, 2008 at 5:00 pm
Rob Daly of Dealing with Technology fame recently wrote a piece in Waters Magazine about the incessant search for faster trade execution and our obsessive quest for the holy grail: low latency. He admonishes us to "Stop the Latency Insanity." Rob then goes on:
But please, people, get a grip-there is only so much that physics can do with most firms' existing architecture...While reducing latency is an ever-present concern for firms, it must be
addressed in a rational manner, balancing the potential return with its
potential costs.
I agree and I disagree. Perhaps that's an indication of insanity. Perhaps there are more ways to look at the latency problem. Perhaps we'll continue this discussion.
I Reckon We Can Wash Away Them Darn Outliers
September 25, 2008 at 2:00 pm
I really like industry events: the intensity; the competitiveness; the pithy debates; the BS; the incredible concentration of opportunity. The good ones have the pulse and cacophony of a third-world bazaar. The evening parties aren’t too bad either, especially when they’re in New York or Las Vegas.
Fortunately, we had High Performance on Wall Street 2008 in New York this week. It was at the Roosevelt Hotel, La Grande Dame from La Belle Époque. I felt out of place without my frock suit and top hat, but mentally returned to the 21st century and enjoyed the healthy debates on in-line risk calculations, low latency, complex event processing, co-lo architecture, etc.
Candidly, with all the market animation this week, it was tough to gauge what the atmosphere would be going in to the event. With rare exceptions (“Hey, where’s the Lehman dude on the panel?”) things went rather smoothly. Hats off to Pete Harris for keeping things lively and moving.
I was able to catch some very good panel discussions as well. I’m always leery of panels because … well … many (regardless of the event) are truly awful. Often panelists are inarticulate, rambling or just too timid to have an opinion. And moderators? They’re running the show but often forget their role and responsibility to us not seated on the dais. Far too typically moderators go through the panelists in sequential order asking softball questions. For most of the audience this proves (often after the ubiquitous high-carb lunch) a quick route to slumberland. When I’m in the audience, I want to be entertained and mentally challenged. I want Jerry Springer meets Harvard. Hey Mr./Ms. Moderator: pick a fight, force a debate, enlighten me and stop wasting my time.
Off my soapbox and back to the panels (because this was not much of an issue at this week’s event). I very much enjoyed the session “Gaining First Mover Advantage with a Low Latency Market Data Solution.” I skipped the concurrent “Low Latency Market Data Distribution – Software and Hardware Approaches” panel because I work with Barry Thompson and get to eat, drink and debate with him all the time. What struck me about the first panel was a very poignant comment made by an excellent panelist. He commented on the market obsession with low latency and the collective failure to account for devastating effect of outliers – late, non-low-latency information. Algos just don’t like outliers and it screws many of them up. Outliers are low-latency impurity. 99.44% pure is good for Ivory Soap, but it’s not good enough for today’s environments.
This subject deserves much more scrutiny. Outliers undermine low latency. Maybe a debate on this at the Roosevelt next year?
OSI, The Waltons and Espresso
September 24, 2008 at 5:00 pm
Every time I hear the phase “OSI Stack” I cringe. Perhaps it’s residual animus from my youthful days at Apollo Computer where those of us doing “second-rate” protocols like TCP/IP and SNA were mocked by elitist colleagues working on the Open Systems Interconnect model. I’m surprised they had time to look down upon us, especially given the demands of going to Europe to attend all the standards meetings. I sat in a dark office, reading RFCs, writing code, drinking crappy coffee and trying to make stuff work over this other “inferior” thing called Ethernet. My colleagues, meanwhile, were sipping espresso on Lake Geneva.
Amazing how time changes everything. My memories were stirred by a recent article in Network World by Steve Taylor and Jim Metzler “Why it's time to let the OSI model die.” I think it’s been six feet under for some time now, but I still think Steve and Jim made their new 3-layer model too complex. Perhaps it’s only 2 layers: application and network. We know company XYZ has an application group and a network group, don’t we? And, like two siblings, those two groups always get along, don't they? Maybe we do need a seven layer model again, somewhat like The Waltons of technology. And they may get along better, too. I’m imagining my new world… “Rob, I think you’ll need to speak with my dear friend in the layer 6 group. I think he’s drinking espresso in the cafeteria with the layer 1 and 3 guys.”
OSI. It’s dead. Go bury it on Walton's Mountain. So, where's middleware? Layer 2.5. At least it's stillbetter than 3.
G’night John-Boy.
More than the Usual Suspects
September 19, 2008 at 2:00 pm
I had the opportunity to attend a great panel discussion at this week’s Gartner Event Processing Summit 2008 in Stamford, CT. It was moderated by Roy Schulte, Gartner VP Distinguished Analyst and included not just the usual suspects in the messaging space (read software), but some of the newer players (read hardware acceleration) including (gratuitous reference) Tervela. Roy was even-handed, yet challenged all of us on our solution approach, differentiation and target market.
So, what was the motivation for this panel entitled Ultra-Low Latency Messaging Panel Discussion? Events, messages and low-latency communications are critical components of what we do. Roy commented that events are becoming more widely deployed in diverse business settings. More importantly, those events have increasing time sensitivity. Messaging, of course, is the foundation for event processing now and moving forward.
So, what was my take? Aside from the typical “some speakers were good and some were less so,” one thing was clear: the world of messaging is splitting into software and hardware. To say one is better than the other would be naïve – and we don’t even position ourselves that way. But one thing we do know: when the volumes start getting high, software systems just can’t keep up. (When was the last time you used a software-based LAN switch?) Is this new? No, but with the rise of events and their underlying messages, we’re starting to redline the performance tachometer.
Where is this all going? Just look to the financial services industry; it’s been a rather good harbinger for the broader enterprise.
LSE Outage and Market(s) Volatility – An “Absolute Nightmare”
September 8, 2008 at 1:00 pm
My phone started ringing early this morning.
“The LSE is down,” said the voice on the line.
“You’re kidding me,” I responded. “Please not today. The Fed announced the Fannie Mae and Freddie Mac rescue. Great day to be in the market.”
“Exactly. Lots of volatility.”
“Any idea of the cause?”
“Nope. It’s been called a ‘connectivity failure’ by the LSE.”
Lots of questions around this outage. I’m sure the LSE is not too happy either.
From the Wall Street Journal:
The connectivity failure by the London Stock Exchange Group PLC, the exchange operator, is an "absolute nightmare," said one trader. "We are just not trading on what should have been a big trading day."
Was it an application failure?
Was it a server crash?
Was it a network failure?
Was it a middleware issue?
We’ll get more information over the coming days, but rest assured we’re seeing the results of building complex trading ecosystems with interdependencies that have their own, often significant risk surface areas. It's not just an LSE issue.
Is there a baseball player in the house? I need some performance-enhancing drugs.
August 27, 2008 at 1:00 pm
A recent article by Fayazuddin Shiraz in Chief Executive turned me onto a post by Jonathan Schwartz, CEO of Sun, lauding the company’s recent announcement of 1 million messages per second for RMDS (Reuters Market Data System). Congratulations to Sun and Intel on the worthwhile collaboration. There is, however, an interesting implication: what happens to the network when you source such a vast amount of market data to a large number of very hungry electronic consumers? (Who may, of course, generate derivative traffic.) Rhetorical question perhaps, but this volume of messaging data can destroy even the most well-provisioned infrastructure regardless of the bandwidth ceiling.
Some time ago (in the pre-steroids era), we had message traffic flowing on a separate network. It was the right thing to do. Later, that network was collapsed into the “new and improved” high performance switched infrastructure. All the data lived happily together for a while – until message volumes went up and multicast trashed the network. And everyone screamed about low latency. Now we’re seeing the pendulum swing back to separate networks for messaging again. We don’t have time to manage one network and soon we’ll have two.
Messaging for the Masses
July 25, 2008 at 11:00 am
I was reviewing some notes from a discussion I had with Kevin McPartland from Tabb Group. We talked about messaging volumes in the global equities and options markets. His numbers were an astonishing 7 billion (Yes, Virginia, that's a "B") messages per day in 2007. More interesting, though, were the projections: 128 billion messages per day by 2010. Wow. And low latency, too.
Street Legal Messaging
July 7, 2008 at 1:20 pm
Marc recently had a very good post on his Magmasystems Blog regarding a recent announcement by a legacy messaging vendor that they were entering the hardware game. As a hardware vendor in the messaging space, we welcome the market validation but we have several questions:
Why is this not akin to a John Kerry flip flop?
Most recent noise on the The Street had said vendor saying no, no, no to hardware. We, of course, argue that it’s inevitable. Look at what the networking world taught us.
Should this legacy messaging be hardware accelerated?
It’s not about messaging; it’s about a messaging architecture that supports acceleration. More lessons here, this time from the automobile world. Shelby SuperCars introduced a 1,183 hp rocket last year that hit 257.41 mph on the road. It wasn’t just about the engine, though. The rest of the car was architecturally matched to it. Putting that engine in the classic Toyota Camry could produce much different – and possibly dangerous - results. You be the judge.
Will this make other problems go away, too?
Again, it’s about architecture. Nothing like a slow messaging consumer to bring down a peer-to-peer framework – even one that’s hardware accelerated. You need a new architecture to fix this. Period.
What about all this XML stuff?
I’ll answer with a question: How prevalent is XML in the high-performance world? Before anyone barks – I’m an XML fan – but for the right applications.
Welcome to the high performance world of hardware. May your engine bolts be fastened properly and your drive train sufficiently matched.
Going the Revolution Route
June 27, 2008 at 11:45 am
OK, we were a bit unorthodox in our approach at SIFMA. When you’re dealing with 7,000 attendees and 300 vendors, you have to be. Perhaps our booth felt like the Haight-Ashbury district of San Francisco in 1967, but we have a big issue with “the establishment” – the messaging establishment. What we’re calling into question is the architecture of legacy messaging systems, message-oriented middleware, etc. Are we stuck with messaging frameworks that are about as contemporary as the tie-dye VW bus? In doubt? Ask a messaging architect about the last outage with a software-based messaging solution. The new revolution is about hardware, architecture and rethinking messaging. It’s time for the message network.
Tags: message network (1) message revolution (1) messaging systems (1) message-oriented middleware (1) haight-ashbury (1)
Comments (0)
You ain’t seen nothin’ yet.
June 20, 2008 at 3:30 pm
Welcome to the Tervela Blog. In our sanitized world of corporate communications, we often have to “play within the lines.” Corporate blogs in particular scare us. Why? Because they tend to be regurgitated marketing speak. If people want that, they’ll go to our web site and get it undigested. So why are we doing this? It’s simple: we believe the messaging industry is at a critical inflection point – and many have strong opinions that we want to discuss and debate. We’d be foolish not to, especially given how strongly we believe messaging is the foundation for future computing platforms, SOA, Web 3.0, etc… (That wasn’t a typo: you be the judge on Web 2.0). Thought the messaging card was played in the 90s? That was the pre-season. You ain’t seen nothin’ yet. We look forward to your thoughts and opinions.
What a way to end the month of October 2008. Let's look at the VIX, Chicago Board Options Exchange Volatility Index.
Rob Daly of Dealing with Technology fame recently wrote a piece in Waters Magazine about the incessant search for faster trade execution and our obsessive quest for the holy grail: low latency. He admonishes us to "Stop the Latency Insanity." Rob then goes on:
But please, people, get a grip-there is only so much that physics can do with most firms' existing architecture...While reducing latency is an ever-present concern for firms, it must be addressed in a rational manner, balancing the potential return with its potential costs.
I agree and I disagree. Perhaps that's an indication of insanity. Perhaps there are more ways to look at the latency problem. Perhaps we'll continue this discussion.
I Reckon We Can Wash Away Them Darn Outliers
September 25, 2008 at 2:00 pm
I really like industry events: the intensity; the competitiveness; the pithy debates; the BS; the incredible concentration of opportunity. The good ones have the pulse and cacophony of a third-world bazaar. The evening parties aren’t too bad either, especially when they’re in New York or Las Vegas.
Fortunately, we had High Performance on Wall Street 2008 in New York this week. It was at the Roosevelt Hotel, La Grande Dame from La Belle Époque. I felt out of place without my frock suit and top hat, but mentally returned to the 21st century and enjoyed the healthy debates on in-line risk calculations, low latency, complex event processing, co-lo architecture, etc.
Candidly, with all the market animation this week, it was tough to gauge what the atmosphere would be going in to the event. With rare exceptions (“Hey, where’s the Lehman dude on the panel?”) things went rather smoothly. Hats off to Pete Harris for keeping things lively and moving.
I was able to catch some very good panel discussions as well. I’m always leery of panels because … well … many (regardless of the event) are truly awful. Often panelists are inarticulate, rambling or just too timid to have an opinion. And moderators? They’re running the show but often forget their role and responsibility to us not seated on the dais. Far too typically moderators go through the panelists in sequential order asking softball questions. For most of the audience this proves (often after the ubiquitous high-carb lunch) a quick route to slumberland. When I’m in the audience, I want to be entertained and mentally challenged. I want Jerry Springer meets Harvard. Hey Mr./Ms. Moderator: pick a fight, force a debate, enlighten me and stop wasting my time.
Off my soapbox and back to the panels (because this was not much of an issue at this week’s event). I very much enjoyed the session “Gaining First Mover Advantage with a Low Latency Market Data Solution.” I skipped the concurrent “Low Latency Market Data Distribution – Software and Hardware Approaches” panel because I work with Barry Thompson and get to eat, drink and debate with him all the time. What struck me about the first panel was a very poignant comment made by an excellent panelist. He commented on the market obsession with low latency and the collective failure to account for devastating effect of outliers – late, non-low-latency information. Algos just don’t like outliers and it screws many of them up. Outliers are low-latency impurity. 99.44% pure is good for Ivory Soap, but it’s not good enough for today’s environments.
This subject deserves much more scrutiny. Outliers undermine low latency. Maybe a debate on this at the Roosevelt next year?
OSI, The Waltons and Espresso
September 24, 2008 at 5:00 pm
Every time I hear the phase “OSI Stack” I cringe. Perhaps it’s residual animus from my youthful days at Apollo Computer where those of us doing “second-rate” protocols like TCP/IP and SNA were mocked by elitist colleagues working on the Open Systems Interconnect model. I’m surprised they had time to look down upon us, especially given the demands of going to Europe to attend all the standards meetings. I sat in a dark office, reading RFCs, writing code, drinking crappy coffee and trying to make stuff work over this other “inferior” thing called Ethernet. My colleagues, meanwhile, were sipping espresso on Lake Geneva.
Amazing how time changes everything. My memories were stirred by a recent article in Network World by Steve Taylor and Jim Metzler “Why it's time to let the OSI model die.” I think it’s been six feet under for some time now, but I still think Steve and Jim made their new 3-layer model too complex. Perhaps it’s only 2 layers: application and network. We know company XYZ has an application group and a network group, don’t we? And, like two siblings, those two groups always get along, don't they? Maybe we do need a seven layer model again, somewhat like The Waltons of technology. And they may get along better, too. I’m imagining my new world… “Rob, I think you’ll need to speak with my dear friend in the layer 6 group. I think he’s drinking espresso in the cafeteria with the layer 1 and 3 guys.”
OSI. It’s dead. Go bury it on Walton's Mountain. So, where's middleware? Layer 2.5. At least it's stillbetter than 3.
G’night John-Boy.
More than the Usual Suspects
September 19, 2008 at 2:00 pm
I had the opportunity to attend a great panel discussion at this week’s Gartner Event Processing Summit 2008 in Stamford, CT. It was moderated by Roy Schulte, Gartner VP Distinguished Analyst and included not just the usual suspects in the messaging space (read software), but some of the newer players (read hardware acceleration) including (gratuitous reference) Tervela. Roy was even-handed, yet challenged all of us on our solution approach, differentiation and target market.
So, what was the motivation for this panel entitled Ultra-Low Latency Messaging Panel Discussion? Events, messages and low-latency communications are critical components of what we do. Roy commented that events are becoming more widely deployed in diverse business settings. More importantly, those events have increasing time sensitivity. Messaging, of course, is the foundation for event processing now and moving forward.
So, what was my take? Aside from the typical “some speakers were good and some were less so,” one thing was clear: the world of messaging is splitting into software and hardware. To say one is better than the other would be naïve – and we don’t even position ourselves that way. But one thing we do know: when the volumes start getting high, software systems just can’t keep up. (When was the last time you used a software-based LAN switch?) Is this new? No, but with the rise of events and their underlying messages, we’re starting to redline the performance tachometer.
Where is this all going? Just look to the financial services industry; it’s been a rather good harbinger for the broader enterprise.
LSE Outage and Market(s) Volatility – An “Absolute Nightmare”
September 8, 2008 at 1:00 pm
My phone started ringing early this morning.
“The LSE is down,” said the voice on the line.
“You’re kidding me,” I responded. “Please not today. The Fed announced the Fannie Mae and Freddie Mac rescue. Great day to be in the market.”
“Exactly. Lots of volatility.”
“Any idea of the cause?”
“Nope. It’s been called a ‘connectivity failure’ by the LSE.”
Lots of questions around this outage. I’m sure the LSE is not too happy either.
From the Wall Street Journal:
The connectivity failure by the London Stock Exchange Group PLC, the exchange operator, is an "absolute nightmare," said one trader. "We are just not trading on what should have been a big trading day."
Was it an application failure?
Was it a server crash?
Was it a network failure?
Was it a middleware issue?
We’ll get more information over the coming days, but rest assured we’re seeing the results of building complex trading ecosystems with interdependencies that have their own, often significant risk surface areas. It's not just an LSE issue.
Is there a baseball player in the house? I need some performance-enhancing drugs.
August 27, 2008 at 1:00 pm
A recent article by Fayazuddin Shiraz in Chief Executive turned me onto a post by Jonathan Schwartz, CEO of Sun, lauding the company’s recent announcement of 1 million messages per second for RMDS (Reuters Market Data System). Congratulations to Sun and Intel on the worthwhile collaboration. There is, however, an interesting implication: what happens to the network when you source such a vast amount of market data to a large number of very hungry electronic consumers? (Who may, of course, generate derivative traffic.) Rhetorical question perhaps, but this volume of messaging data can destroy even the most well-provisioned infrastructure regardless of the bandwidth ceiling.
Some time ago (in the pre-steroids era), we had message traffic flowing on a separate network. It was the right thing to do. Later, that network was collapsed into the “new and improved” high performance switched infrastructure. All the data lived happily together for a while – until message volumes went up and multicast trashed the network. And everyone screamed about low latency. Now we’re seeing the pendulum swing back to separate networks for messaging again. We don’t have time to manage one network and soon we’ll have two.
Messaging for the Masses
July 25, 2008 at 11:00 am
I was reviewing some notes from a discussion I had with Kevin McPartland from Tabb Group. We talked about messaging volumes in the global equities and options markets. His numbers were an astonishing 7 billion (Yes, Virginia, that's a "B") messages per day in 2007. More interesting, though, were the projections: 128 billion messages per day by 2010. Wow. And low latency, too.
Street Legal Messaging
July 7, 2008 at 1:20 pm
Marc recently had a very good post on his Magmasystems Blog regarding a recent announcement by a legacy messaging vendor that they were entering the hardware game. As a hardware vendor in the messaging space, we welcome the market validation but we have several questions:
Why is this not akin to a John Kerry flip flop?
Most recent noise on the The Street had said vendor saying no, no, no to hardware. We, of course, argue that it’s inevitable. Look at what the networking world taught us.
Should this legacy messaging be hardware accelerated?
It’s not about messaging; it’s about a messaging architecture that supports acceleration. More lessons here, this time from the automobile world. Shelby SuperCars introduced a 1,183 hp rocket last year that hit 257.41 mph on the road. It wasn’t just about the engine, though. The rest of the car was architecturally matched to it. Putting that engine in the classic Toyota Camry could produce much different – and possibly dangerous - results. You be the judge.
Will this make other problems go away, too?
Again, it’s about architecture. Nothing like a slow messaging consumer to bring down a peer-to-peer framework – even one that’s hardware accelerated. You need a new architecture to fix this. Period.
What about all this XML stuff?
I’ll answer with a question: How prevalent is XML in the high-performance world? Before anyone barks – I’m an XML fan – but for the right applications.
Welcome to the high performance world of hardware. May your engine bolts be fastened properly and your drive train sufficiently matched.
Going the Revolution Route
June 27, 2008 at 11:45 am
OK, we were a bit unorthodox in our approach at SIFMA. When you’re dealing with 7,000 attendees and 300 vendors, you have to be. Perhaps our booth felt like the Haight-Ashbury district of San Francisco in 1967, but we have a big issue with “the establishment” – the messaging establishment. What we’re calling into question is the architecture of legacy messaging systems, message-oriented middleware, etc. Are we stuck with messaging frameworks that are about as contemporary as the tie-dye VW bus? In doubt? Ask a messaging architect about the last outage with a software-based messaging solution. The new revolution is about hardware, architecture and rethinking messaging. It’s time for the message network.
Tags: message network (1) message revolution (1) messaging systems (1) message-oriented middleware (1) haight-ashbury (1)
Comments (0)
You ain’t seen nothin’ yet.
June 20, 2008 at 3:30 pm
Welcome to the Tervela Blog. In our sanitized world of corporate communications, we often have to “play within the lines.” Corporate blogs in particular scare us. Why? Because they tend to be regurgitated marketing speak. If people want that, they’ll go to our web site and get it undigested. So why are we doing this? It’s simple: we believe the messaging industry is at a critical inflection point – and many have strong opinions that we want to discuss and debate. We’d be foolish not to, especially given how strongly we believe messaging is the foundation for future computing platforms, SOA, Web 3.0, etc… (That wasn’t a typo: you be the judge on Web 2.0). Thought the messaging card was played in the 90s? That was the pre-season. You ain’t seen nothin’ yet. We look forward to your thoughts and opinions.
I really like industry events: the intensity; the competitiveness; the pithy debates; the BS; the incredible concentration of opportunity. The good ones have the pulse and cacophony of a third-world bazaar. The evening parties aren’t too bad either, especially when they’re in New York or Las Vegas.
Fortunately, we had High Performance on Wall Street 2008 in New York this week. It was at the Roosevelt Hotel, La Grande Dame from La Belle Époque. I felt out of place without my frock suit and top hat, but mentally returned to the 21st century and enjoyed the healthy debates on in-line risk calculations, low latency, complex event processing, co-lo architecture, etc.
Candidly, with all the market animation this week, it was tough to gauge what the atmosphere would be going in to the event. With rare exceptions (“Hey, where’s the Lehman dude on the panel?”) things went rather smoothly. Hats off to Pete Harris for keeping things lively and moving.
I was able to catch some very good panel discussions as well. I’m always leery of panels because … well … many (regardless of the event) are truly awful. Often panelists are inarticulate, rambling or just too timid to have an opinion. And moderators? They’re running the show but often forget their role and responsibility to us not seated on the dais. Far too typically moderators go through the panelists in sequential order asking softball questions. For most of the audience this proves (often after the ubiquitous high-carb lunch) a quick route to slumberland. When I’m in the audience, I want to be entertained and mentally challenged. I want Jerry Springer meets Harvard. Hey Mr./Ms. Moderator: pick a fight, force a debate, enlighten me and stop wasting my time.
Off my soapbox and back to the panels (because this was not much of an issue at this week’s event). I very much enjoyed the session “Gaining First Mover Advantage with a Low Latency Market Data Solution.” I skipped the concurrent “Low Latency Market Data Distribution – Software and Hardware Approaches” panel because I work with Barry Thompson and get to eat, drink and debate with him all the time. What struck me about the first panel was a very poignant comment made by an excellent panelist. He commented on the market obsession with low latency and the collective failure to account for devastating effect of outliers – late, non-low-latency information. Algos just don’t like outliers and it screws many of them up. Outliers are low-latency impurity. 99.44% pure is good for Ivory Soap, but it’s not good enough for today’s environments.
This subject deserves much more scrutiny. Outliers undermine low latency. Maybe a debate on this at the Roosevelt next year?
Every time I hear the phase “OSI Stack” I cringe. Perhaps it’s residual animus from my youthful days at Apollo Computer where those of us doing “second-rate” protocols like TCP/IP and SNA were mocked by elitist colleagues working on the Open Systems Interconnect model. I’m surprised they had time to look down upon us, especially given the demands of going to Europe to attend all the standards meetings. I sat in a dark office, reading RFCs, writing code, drinking crappy coffee and trying to make stuff work over this other “inferior” thing called Ethernet. My colleagues, meanwhile, were sipping espresso on Lake Geneva.
Amazing how time changes everything. My memories were stirred by a recent article in Network World by Steve Taylor and Jim Metzler “Why it's time to let the OSI model die.” I think it’s been six feet under for some time now, but I still think Steve and Jim made their new 3-layer model too complex. Perhaps it’s only 2 layers: application and network. We know company XYZ has an application group and a network group, don’t we? And, like two siblings, those two groups always get along, don't they? Maybe we do need a seven layer model again, somewhat like The Waltons of technology. And they may get along better, too. I’m imagining my new world… “Rob, I think you’ll need to speak with my dear friend in the layer 6 group. I think he’s drinking espresso in the cafeteria with the layer 1 and 3 guys.”
OSI. It’s dead. Go bury it on Walton's Mountain. So, where's middleware? Layer 2.5. At least it's stillbetter than 3.
G’night John-Boy.
More than the Usual Suspects
September 19, 2008 at 2:00 pm
I had the opportunity to attend a great panel discussion at this week’s Gartner Event Processing Summit 2008 in Stamford, CT. It was moderated by Roy Schulte, Gartner VP Distinguished Analyst and included not just the usual suspects in the messaging space (read software), but some of the newer players (read hardware acceleration) including (gratuitous reference) Tervela. Roy was even-handed, yet challenged all of us on our solution approach, differentiation and target market.
So, what was the motivation for this panel entitled Ultra-Low Latency Messaging Panel Discussion? Events, messages and low-latency communications are critical components of what we do. Roy commented that events are becoming more widely deployed in diverse business settings. More importantly, those events have increasing time sensitivity. Messaging, of course, is the foundation for event processing now and moving forward.
So, what was my take? Aside from the typical “some speakers were good and some were less so,” one thing was clear: the world of messaging is splitting into software and hardware. To say one is better than the other would be naïve – and we don’t even position ourselves that way. But one thing we do know: when the volumes start getting high, software systems just can’t keep up. (When was the last time you used a software-based LAN switch?) Is this new? No, but with the rise of events and their underlying messages, we’re starting to redline the performance tachometer.
Where is this all going? Just look to the financial services industry; it’s been a rather good harbinger for the broader enterprise.
LSE Outage and Market(s) Volatility – An “Absolute Nightmare”
September 8, 2008 at 1:00 pm
My phone started ringing early this morning.
“The LSE is down,” said the voice on the line.
“You’re kidding me,” I responded. “Please not today. The Fed announced the Fannie Mae and Freddie Mac rescue. Great day to be in the market.”
“Exactly. Lots of volatility.”
“Any idea of the cause?”
“Nope. It’s been called a ‘connectivity failure’ by the LSE.”
Lots of questions around this outage. I’m sure the LSE is not too happy either.
From the Wall Street Journal:
The connectivity failure by the London Stock Exchange Group PLC, the exchange operator, is an "absolute nightmare," said one trader. "We are just not trading on what should have been a big trading day."
Was it an application failure?
Was it a server crash?
Was it a network failure?
Was it a middleware issue?
We’ll get more information over the coming days, but rest assured we’re seeing the results of building complex trading ecosystems with interdependencies that have their own, often significant risk surface areas. It's not just an LSE issue.
Is there a baseball player in the house? I need some performance-enhancing drugs.
August 27, 2008 at 1:00 pm
A recent article by Fayazuddin Shiraz in Chief Executive turned me onto a post by Jonathan Schwartz, CEO of Sun, lauding the company’s recent announcement of 1 million messages per second for RMDS (Reuters Market Data System). Congratulations to Sun and Intel on the worthwhile collaboration. There is, however, an interesting implication: what happens to the network when you source such a vast amount of market data to a large number of very hungry electronic consumers? (Who may, of course, generate derivative traffic.) Rhetorical question perhaps, but this volume of messaging data can destroy even the most well-provisioned infrastructure regardless of the bandwidth ceiling.
Some time ago (in the pre-steroids era), we had message traffic flowing on a separate network. It was the right thing to do. Later, that network was collapsed into the “new and improved” high performance switched infrastructure. All the data lived happily together for a while – until message volumes went up and multicast trashed the network. And everyone screamed about low latency. Now we’re seeing the pendulum swing back to separate networks for messaging again. We don’t have time to manage one network and soon we’ll have two.
Messaging for the Masses
July 25, 2008 at 11:00 am
I was reviewing some notes from a discussion I had with Kevin McPartland from Tabb Group. We talked about messaging volumes in the global equities and options markets. His numbers were an astonishing 7 billion (Yes, Virginia, that's a "B") messages per day in 2007. More interesting, though, were the projections: 128 billion messages per day by 2010. Wow. And low latency, too.
Street Legal Messaging
July 7, 2008 at 1:20 pm
Marc recently had a very good post on his Magmasystems Blog regarding a recent announcement by a legacy messaging vendor that they were entering the hardware game. As a hardware vendor in the messaging space, we welcome the market validation but we have several questions:
Why is this not akin to a John Kerry flip flop?
Most recent noise on the The Street had said vendor saying no, no, no to hardware. We, of course, argue that it’s inevitable. Look at what the networking world taught us.
Should this legacy messaging be hardware accelerated?
It’s not about messaging; it’s about a messaging architecture that supports acceleration. More lessons here, this time from the automobile world. Shelby SuperCars introduced a 1,183 hp rocket last year that hit 257.41 mph on the road. It wasn’t just about the engine, though. The rest of the car was architecturally matched to it. Putting that engine in the classic Toyota Camry could produce much different – and possibly dangerous - results. You be the judge.
Will this make other problems go away, too?
Again, it’s about architecture. Nothing like a slow messaging consumer to bring down a peer-to-peer framework – even one that’s hardware accelerated. You need a new architecture to fix this. Period.
What about all this XML stuff?
I’ll answer with a question: How prevalent is XML in the high-performance world? Before anyone barks – I’m an XML fan – but for the right applications.
Welcome to the high performance world of hardware. May your engine bolts be fastened properly and your drive train sufficiently matched.
Going the Revolution Route
June 27, 2008 at 11:45 am
OK, we were a bit unorthodox in our approach at SIFMA. When you’re dealing with 7,000 attendees and 300 vendors, you have to be. Perhaps our booth felt like the Haight-Ashbury district of San Francisco in 1967, but we have a big issue with “the establishment” – the messaging establishment. What we’re calling into question is the architecture of legacy messaging systems, message-oriented middleware, etc. Are we stuck with messaging frameworks that are about as contemporary as the tie-dye VW bus? In doubt? Ask a messaging architect about the last outage with a software-based messaging solution. The new revolution is about hardware, architecture and rethinking messaging. It’s time for the message network.
Tags: message network (1) message revolution (1) messaging systems (1) message-oriented middleware (1) haight-ashbury (1)
Comments (0)
You ain’t seen nothin’ yet.
June 20, 2008 at 3:30 pm
Welcome to the Tervela Blog. In our sanitized world of corporate communications, we often have to “play within the lines.” Corporate blogs in particular scare us. Why? Because they tend to be regurgitated marketing speak. If people want that, they’ll go to our web site and get it undigested. So why are we doing this? It’s simple: we believe the messaging industry is at a critical inflection point – and many have strong opinions that we want to discuss and debate. We’d be foolish not to, especially given how strongly we believe messaging is the foundation for future computing platforms, SOA, Web 3.0, etc… (That wasn’t a typo: you be the judge on Web 2.0). Thought the messaging card was played in the 90s? That was the pre-season. You ain’t seen nothin’ yet. We look forward to your thoughts and opinions.
I had the opportunity to attend a great panel discussion at this week’s Gartner Event Processing Summit 2008 in Stamford, CT. It was moderated by Roy Schulte, Gartner VP Distinguished Analyst and included not just the usual suspects in the messaging space (read software), but some of the newer players (read hardware acceleration) including (gratuitous reference) Tervela. Roy was even-handed, yet challenged all of us on our solution approach, differentiation and target market.
So, what was the motivation for this panel entitled Ultra-Low Latency Messaging Panel Discussion? Events, messages and low-latency communications are critical components of what we do. Roy commented that events are becoming more widely deployed in diverse business settings. More importantly, those events have increasing time sensitivity. Messaging, of course, is the foundation for event processing now and moving forward.
So, what was my take? Aside from the typical “some speakers were good and some were less so,” one thing was clear: the world of messaging is splitting into software and hardware. To say one is better than the other would be naïve – and we don’t even position ourselves that way. But one thing we do know: when the volumes start getting high, software systems just can’t keep up. (When was the last time you used a software-based LAN switch?) Is this new? No, but with the rise of events and their underlying messages, we’re starting to redline the performance tachometer.
Where is this all going? Just look to the financial services industry; it’s been a rather good harbinger for the broader enterprise.
My phone started ringing early this morning.
“The LSE is down,” said the voice on the line.
“You’re kidding me,” I responded. “Please not today. The Fed announced the Fannie Mae and Freddie Mac rescue. Great day to be in the market.”
“Exactly. Lots of volatility.”
“Any idea of the cause?”
“Nope. It’s been called a ‘connectivity failure’ by the LSE.”
Lots of questions around this outage. I’m sure the LSE is not too happy either.
From the Wall Street Journal:
The connectivity failure by the London Stock Exchange Group PLC, the exchange operator, is an "absolute nightmare," said one trader. "We are just not trading on what should have been a big trading day."
Was it an application failure?
Was it a server crash?
Was it a network failure?
Was it a middleware issue?
We’ll get more information over the coming days, but rest assured we’re seeing the results of building complex trading ecosystems with interdependencies that have their own, often significant risk surface areas. It's not just an LSE issue.
Is there a baseball player in the house? I need some performance-enhancing drugs.
August 27, 2008 at 1:00 pm
A recent article by Fayazuddin Shiraz in Chief Executive turned me onto a post by Jonathan Schwartz, CEO of Sun, lauding the company’s recent announcement of 1 million messages per second for RMDS (Reuters Market Data System). Congratulations to Sun and Intel on the worthwhile collaboration. There is, however, an interesting implication: what happens to the network when you source such a vast amount of market data to a large number of very hungry electronic consumers? (Who may, of course, generate derivative traffic.) Rhetorical question perhaps, but this volume of messaging data can destroy even the most well-provisioned infrastructure regardless of the bandwidth ceiling.
Some time ago (in the pre-steroids era), we had message traffic flowing on a separate network. It was the right thing to do. Later, that network was collapsed into the “new and improved” high performance switched infrastructure. All the data lived happily together for a while – until message volumes went up and multicast trashed the network. And everyone screamed about low latency. Now we’re seeing the pendulum swing back to separate networks for messaging again. We don’t have time to manage one network and soon we’ll have two.
Messaging for the Masses
July 25, 2008 at 11:00 am
I was reviewing some notes from a discussion I had with Kevin McPartland from Tabb Group. We talked about messaging volumes in the global equities and options markets. His numbers were an astonishing 7 billion (Yes, Virginia, that's a "B") messages per day in 2007. More interesting, though, were the projections: 128 billion messages per day by 2010. Wow. And low latency, too.
Street Legal Messaging
July 7, 2008 at 1:20 pm
Marc recently had a very good post on his Magmasystems Blog regarding a recent announcement by a legacy messaging vendor that they were entering the hardware game. As a hardware vendor in the messaging space, we welcome the market validation but we have several questions:
Why is this not akin to a John Kerry flip flop?
Most recent noise on the The Street had said vendor saying no, no, no to hardware. We, of course, argue that it’s inevitable. Look at what the networking world taught us.
Should this legacy messaging be hardware accelerated?
It’s not about messaging; it’s about a messaging architecture that supports acceleration. More lessons here, this time from the automobile world. Shelby SuperCars introduced a 1,183 hp rocket last year that hit 257.41 mph on the road. It wasn’t just about the engine, though. The rest of the car was architecturally matched to it. Putting that engine in the classic Toyota Camry could produce much different – and possibly dangerous - results. You be the judge.
Will this make other problems go away, too?
Again, it’s about architecture. Nothing like a slow messaging consumer to bring down a peer-to-peer framework – even one that’s hardware accelerated. You need a new architecture to fix this. Period.
What about all this XML stuff?
I’ll answer with a question: How prevalent is XML in the high-performance world? Before anyone barks – I’m an XML fan – but for the right applications.
Welcome to the high performance world of hardware. May your engine bolts be fastened properly and your drive train sufficiently matched.
Going the Revolution Route
June 27, 2008 at 11:45 am
OK, we were a bit unorthodox in our approach at SIFMA. When you’re dealing with 7,000 attendees and 300 vendors, you have to be. Perhaps our booth felt like the Haight-Ashbury district of San Francisco in 1967, but we have a big issue with “the establishment” – the messaging establishment. What we’re calling into question is the architecture of legacy messaging systems, message-oriented middleware, etc. Are we stuck with messaging frameworks that are about as contemporary as the tie-dye VW bus? In doubt? Ask a messaging architect about the last outage with a software-based messaging solution. The new revolution is about hardware, architecture and rethinking messaging. It’s time for the message network.
Tags: message network (1) message revolution (1) messaging systems (1) message-oriented middleware (1) haight-ashbury (1)
Comments (0)
You ain’t seen nothin’ yet.
June 20, 2008 at 3:30 pm
Welcome to the Tervela Blog. In our sanitized world of corporate communications, we often have to “play within the lines.” Corporate blogs in particular scare us. Why? Because they tend to be regurgitated marketing speak. If people want that, they’ll go to our web site and get it undigested. So why are we doing this? It’s simple: we believe the messaging industry is at a critical inflection point – and many have strong opinions that we want to discuss and debate. We’d be foolish not to, especially given how strongly we believe messaging is the foundation for future computing platforms, SOA, Web 3.0, etc… (That wasn’t a typo: you be the judge on Web 2.0). Thought the messaging card was played in the 90s? That was the pre-season. You ain’t seen nothin’ yet. We look forward to your thoughts and opinions.
A recent article by Fayazuddin Shiraz in Chief Executive turned me onto a post by Jonathan Schwartz, CEO of Sun, lauding the company’s recent announcement of 1 million messages per second for RMDS (Reuters Market Data System). Congratulations to Sun and Intel on the worthwhile collaboration. There is, however, an interesting implication: what happens to the network when you source such a vast amount of market data to a large number of very hungry electronic consumers? (Who may, of course, generate derivative traffic.) Rhetorical question perhaps, but this volume of messaging data can destroy even the most well-provisioned infrastructure regardless of the bandwidth ceiling.
Some time ago (in the pre-steroids era), we had message traffic flowing on a separate network. It was the right thing to do. Later, that network was collapsed into the “new and improved” high performance switched infrastructure. All the data lived happily together for a while – until message volumes went up and multicast trashed the network. And everyone screamed about low latency. Now we’re seeing the pendulum swing back to separate networks for messaging again. We don’t have time to manage one network and soon we’ll have two.
I was reviewing some notes from a discussion I had with Kevin McPartland from Tabb Group. We talked about messaging volumes in the global equities and options markets. His numbers were an astonishing 7 billion (Yes, Virginia, that's a "B") messages per day in 2007. More interesting, though, were the projections: 128 billion messages per day by 2010. Wow. And low latency, too.
Street Legal Messaging
July 7, 2008 at 1:20 pm
Marc recently had a very good post on his Magmasystems Blog regarding a recent announcement by a legacy messaging vendor that they were entering the hardware game. As a hardware vendor in the messaging space, we welcome the market validation but we have several questions:
Why is this not akin to a John Kerry flip flop?
Most recent noise on the The Street had said vendor saying no, no, no to hardware. We, of course, argue that it’s inevitable. Look at what the networking world taught us.
Should this legacy messaging be hardware accelerated?
It’s not about messaging; it’s about a messaging architecture that supports acceleration. More lessons here, this time from the automobile world. Shelby SuperCars introduced a 1,183 hp rocket last year that hit 257.41 mph on the road. It wasn’t just about the engine, though. The rest of the car was architecturally matched to it. Putting that engine in the classic Toyota Camry could produce much different – and possibly dangerous - results. You be the judge.
Will this make other problems go away, too?
Again, it’s about architecture. Nothing like a slow messaging consumer to bring down a peer-to-peer framework – even one that’s hardware accelerated. You need a new architecture to fix this. Period.
What about all this XML stuff?
I’ll answer with a question: How prevalent is XML in the high-performance world? Before anyone barks – I’m an XML fan – but for the right applications.
Welcome to the high performance world of hardware. May your engine bolts be fastened properly and your drive train sufficiently matched.
Going the Revolution Route
June 27, 2008 at 11:45 am
OK, we were a bit unorthodox in our approach at SIFMA. When you’re dealing with 7,000 attendees and 300 vendors, you have to be. Perhaps our booth felt like the Haight-Ashbury district of San Francisco in 1967, but we have a big issue with “the establishment” – the messaging establishment. What we’re calling into question is the architecture of legacy messaging systems, message-oriented middleware, etc. Are we stuck with messaging frameworks that are about as contemporary as the tie-dye VW bus? In doubt? Ask a messaging architect about the last outage with a software-based messaging solution. The new revolution is about hardware, architecture and rethinking messaging. It’s time for the message network.
Tags: message network (1) message revolution (1) messaging systems (1) message-oriented middleware (1) haight-ashbury (1)
Comments (0)
You ain’t seen nothin’ yet.
June 20, 2008 at 3:30 pm
Welcome to the Tervela Blog. In our sanitized world of corporate communications, we often have to “play within the lines.” Corporate blogs in particular scare us. Why? Because they tend to be regurgitated marketing speak. If people want that, they’ll go to our web site and get it undigested. So why are we doing this? It’s simple: we believe the messaging industry is at a critical inflection point – and many have strong opinions that we want to discuss and debate. We’d be foolish not to, especially given how strongly we believe messaging is the foundation for future computing platforms, SOA, Web 3.0, etc… (That wasn’t a typo: you be the judge on Web 2.0). Thought the messaging card was played in the 90s? That was the pre-season. You ain’t seen nothin’ yet. We look forward to your thoughts and opinions.
Marc recently had a very good post on his Magmasystems Blog regarding a recent announcement by a legacy messaging vendor that they were entering the hardware game. As a hardware vendor in the messaging space, we welcome the market validation but we have several questions:
Why is this not akin to a John Kerry flip flop?
Most recent noise on the The Street had said vendor saying no, no, no to hardware. We, of course, argue that it’s inevitable. Look at what the networking world taught us.
Should this legacy messaging be hardware accelerated?
It’s not about messaging; it’s about a messaging architecture that supports acceleration. More lessons here, this time from the automobile world. Shelby SuperCars introduced a 1,183 hp rocket last year that hit 257.41 mph on the road. It wasn’t just about the engine, though. The rest of the car was architecturally matched to it. Putting that engine in the classic Toyota Camry could produce much different – and possibly dangerous - results. You be the judge.
Will this make other problems go away, too?
Again, it’s about architecture. Nothing like a slow messaging consumer to bring down a peer-to-peer framework – even one that’s hardware accelerated. You need a new architecture to fix this. Period.
What about all this XML stuff?
I’ll answer with a question: How prevalent is XML in the high-performance world? Before anyone barks – I’m an XML fan – but for the right applications.
Welcome to the high performance world of hardware. May your engine bolts be fastened properly and your drive train sufficiently matched.
OK, we were a bit unorthodox in our approach at SIFMA. When you’re dealing with 7,000 attendees and 300 vendors, you have to be. Perhaps our booth felt like the Haight-Ashbury district of San Francisco in 1967, but we have a big issue with “the establishment” – the messaging establishment. What we’re calling into question is the architecture of legacy messaging systems, message-oriented middleware, etc. Are we stuck with messaging frameworks that are about as contemporary as the tie-dye VW bus? In doubt? Ask a messaging architect about the last outage with a software-based messaging solution. The new revolution is about hardware, architecture and rethinking messaging. It’s time for the message network.
Tags: message network (1) message revolution (1) messaging systems (1) message-oriented middleware (1) haight-ashbury (1)
Comments (0) | | | | | |
You ain’t seen nothin’ yet.
June 20, 2008 at 3:30 pm
Welcome to the Tervela Blog. In our sanitized world of corporate communications, we often have to “play within the lines.” Corporate blogs in particular scare us. Why? Because they tend to be regurgitated marketing speak. If people want that, they’ll go to our web site and get it undigested. So why are we doing this? It’s simple: we believe the messaging industry is at a critical inflection point – and many have strong opinions that we want to discuss and debate. We’d be foolish not to, especially given how strongly we believe messaging is the foundation for future computing platforms, SOA, Web 3.0, etc… (That wasn’t a typo: you be the judge on Web 2.0). Thought the messaging card was played in the 90s? That was the pre-season. You ain’t seen nothin’ yet. We look forward to your thoughts and opinions.
Welcome to the Tervela Blog. In our sanitized world of corporate communications, we often have to “play within the lines.” Corporate blogs in particular scare us. Why? Because they tend to be regurgitated marketing speak. If people want that, they’ll go to our web site and get it undigested. So why are we doing this? It’s simple: we believe the messaging industry is at a critical inflection point – and many have strong opinions that we want to discuss and debate. We’d be foolish not to, especially given how strongly we believe messaging is the foundation for future computing platforms, SOA, Web 3.0, etc… (That wasn’t a typo: you be the judge on Web 2.0). Thought the messaging card was played in the 90s? That was the pre-season. You ain’t seen nothin’ yet. We look forward to your thoughts and opinions.
