As you've probably read by now, 30MB versions of the Zune stopped working everywhere around 1:30am early this morning. It's a good reminder that companies need to think about what happens when the cloud goes down. Some say "but local servers go down too." True, but in a DR setup, your environment can withstand losing one datacenter. I know, I've architected a DR site that kept running when one datacenter was lost due to massive power and generator failures following a hurricane. Usage of the cloud needs to be thought of in those terms. You can't rely on just one cloud -- or else expect your business interruption insurance to go after your forced to a file a claim.
There have been countless articles written upon SOA failure rates. But how do you define SOA failure? Sure, it's easy when the project is scrapped. But what about the cases in which the SOA project is completed but way overbudget? Success or failure? Most organizations simply aren't mature enough to define the success or failure of a project. Rather, they use the famous Supreme Court "I know it when I see it" measure. I recommend to our customers that quantitative metrics be used to determine success and failure. It's almost always met with resistance, but defining these success/failure metrics helps bring far-fetched expectations into realtiy and results in a better planned project.
BroadSoft and Sylantro, two struggling VOIP companies failing to gain market share in the application mashup space, are now one. Both platforms will be supported until they are merged. BroadSoft announced this summer that they were moving from SOAP based to RESTful APIs. Sylantro still uses SOAP based APIs. I don't think this acquisition matters -- both play in a space that simply has no interest.
ZDNet has published reader comments on SOA. We disagree with Kirstan who states "Open source tools like XAware for data services, ActiveEndpoints for orchestration, Mule or ServiceMix for ESB, are quite mature." Quite the opposite is true, these tools are immature. Kirstan also adds "open source tools make even greater sense for ‘project level SOA,’ where a company can build an SOA in the small before committing to an expensive enterprise-level SOA." Again, this is bad advice. Let's say you decide to use Mule for "project level SOA". Switching to an enterprise level ESB such as DataPower or Tibco is non-trivial. It is always best practive to develop on the product stack to which you will deploy production.
Joe McKendrick has a nice article highlighting SOA success stories. My favorite example: "To speed up the pace of mergers and acquisitions." This is an often overlooked benefit of SOA -- any company with an SOA-based infrastructure can more easily assimilate an acquired company. Companies often overlook this benefit as they simply aren't mature enough to account for this potential benefit (unlike their ability to account for opportunity costs).
KPI has seen very little activity in staffing SOA projects during the holidays.  Traffic is generally slow this time of year, but this year seems unusually slow.  Perhaps it's the calendar, which has placed both Christmas and New Years on a Thursday (many people have taken 24th-1st off).  KPI has had only one SOA inquiry in the past four days -- I will let you know when traffic picks up.
Amazon is the king of SOA. To understand why, read this article with their CTO Werner Vogels. The interesting points: (1) most Amazon software is homegrown (2) developers have no lock-in, i.e. they aren't forced to use Amazon storage, (3) Shifting investment from up-front to usage-based investment is something that CFOs at this moment clearly love. On point three, I wonder if CFO's consider the costs of the Amazon cloud going down. KPI recommends that organizations not run their entire business on an external cloud for just this reason. We've always maintained that a hybrid approach is best (such as using a cloud to handle traffic peaks).
Joe McKendrick makes five SOA predictions for 2009. I disagree that there will be less hype about SOA. Given the current economy, vendors will likely push SOA even harder. SOA is powering the next wave of corporate technology upgrades -- the more hype, the easier a product is to sell. No company wants to fall behind, even in a bad economy. Expect vendors to play companies against each other in order to drive SOA business.
I have blogged a number of times about the advantages of Amazon's Web Services.  What's missing from this space is a guide on migrating to their services. ZDNet has a useful how-to online, using HelpStream as a use case. The article also shows that determining ROI on SOA, a topic which is discussed endlessly, is not difficult to compute.
John Andrews, CEO of researcher Evans Data Corp, says "During the economic situation we're in now, I expect SOA implementation velocity will decrease. John K. Waters adds "SOA projects are particularly at risk of being cut from budgets due to the difficulty in determining the return on investment (ROI) of such projects." It's not that difficult. If an organization cannot determine the ROI of an SOA project, it is not mature enough to commence a complex SOA project in the first place.
...think again! Micro Focus has announced "Micro Focus COBOL for Eclipse" which allow you to expose COBOL program interfaces as Web Services. Hopefully, organizations will use this as an intermediate step in moving from COBOL to modern technology. We've seen many companies use Progress to connect to legacy system. Some have no intention of moving off the legacy, stating this keeps short term costs down. The forward thinkers use Progress to connect to legacy on an interim basis as they move off the mainframe. The number of COBOL developers is obviously dwindling every year.
Unisys contacted my company a little bit ago about an SOA project but we could not agree on a contract. We were skeptical of their SOA commitment but we were wrong. They will help New Zealand financial services cooperative PSIS go SOA. It's a 4 year commitment. The financial industry has embraced SOA worldwide. With so many recent mergers, SOA can provide for easier integration of disparate systems.
Web Services from the US Postal Services have been knocked offline several times this week due to heavy traffic and unknown computer glitches (bugs I assume).  Click-N-Ship, the most popular Web Service, was  affected.  Many sites had problems this holiday season with Sears.com  (I didn't know people still shopped there) being mostly unavailable on Black Friday.
Forbes.com will use Xignite to provide real-time stock quotes throughout Forbes.com using BATS Exchange trading data. Xignite's cloud services platform and simple Web widgets allows Forbes to integrate real-time stock quotes and charts anywhere in just minutes. The cost is significantly lower cost than traditional market data solutions. Another win-win for cloud computing.
Amazon Web Services logoAmazon is putting public data sets on the cloud. Many researchers spend a good portion of their grant money on IT, reducing the amount of money they spend on their core research. But after a few researchers from Harvard began using Amazon for servers and storage, Amazon got smart. They are now making large data sets available for free -- some of which are so large they would take hours to download. Amazon is hoping that researchers won't download the data, but rather access the data while paying for services. The US Census data is already online. Other data, such as historical economic data, is soon to come.
AmberPointAmberpoint has increased their coverage of the Microsoft platform, including Microsoft Windows Communication Foundation (WCF) and the Microsoft Universal Description, Discovery, and Integration (UDDI) v3 Registry. It's better late than never for Microsoft, which has been unfashionably late to the SOA party. Having the support of vendors such as AmberPoint is crucial to adoption. AmberPoint now has a broad range of support for Microsoft (IIS, Active Directory, Windows Server, SQL Server, BizTalk Server, Operations Manager).
Google
No SoftwareSalesforce.comCRM The World's Favorite Customer Relationship Management - Salesforce.com is now partnerinCRM The World's Favorite Customer Relationship Management - Salesforce.comg with Google to allow its Force.com PAAS access to Google’s cloud-based application development platform. Salesforce develoCRM The World's Favorite Customer Relationship Management - Salesforce.compers will have native access to Google’s distributed storage system (Bigtable) while Google App Engine developers will have access to the Salesforce platform. Google and Salesforce originally teamed up in June 2008, when they announced the release of the Force.com Tool for Google APIs, giving Force.com access to the data in Google Apps. Salesforce.com already connects to Amazon's web services. It's a win-win for both companies.
Quova provides location services that map IP addresses to geographic locations. Currently, customers host a local Quova server to retrieve this mapping. Using Quova's new Web Service, customers can now use a simple script to access the data. Sophisticated e-retailers use Quova to detect possible fraud.  For example, some retailers check the distance between the actual and expected user location. They create rules in their Web applications to decline or review orders falling X miles or more away from the shipping or billing address. E-tailers report that orders coming from 500 miles or more away from the expected location have a higher probability of being fraudulent.
One of the biggest misconceptions I hear from companies is that moving to an SOA based approach will help a company save money in the recession. There are a couple things wrong with this point of view. Of course, SOA is a long term investment which will actually result in higher short term costs. The payoff is long term. The basis for this misconception is rooted in a company's belief that you can build a production ready SOA system in 4-6 months. Believe it or not, we field this request far, far too often. If you're reading this blog, we do not need to explain further.
Virgin Atlantic has renewed an application services deal with EDS for five years. VA, of course, has worked with EDS for 25 years. EDS will develop AirlineSOA "to provide improved customer services, greater agility and lower costs”, according to IT director Mike Cope. American also uses SOA I believe.
Joe McKendrick writes that "Mashups can be done now, without waiting for SOA to come along." I'm not sure I agree with that. Mashups are ideal for SOA. In fact, most mashups use SOA -- just RESTful services, not SOAP based ones. Comments?
Zenbe is a new web service that is a cloud on top of a cloud. Zenbe consolidates all your webmail (Yahoo, GMail, Hotmail, etc) into one. In addition, it offers an online calendar, lists, mobile sync, and file sharing. It also works with Facebook and Twitter. I've tested this and it's a great service...if you trust Zenbe with your e-mails that is.
See full size imageIn a short sighted move, Amazon is shutting
down its Alexa Web Service, one of our favorites. You have until January 26 to move to similar services provided by Yahoo and Google. Amazon cited low usage as the reason. But why would they give users any reason to use a competing service?
GetGlue.com is a relatively new Web Service, available as a browser plug-in that connects you and other Glue users with the sites they visit. After using this for a couple of weeks, Glue's strength seems to be in connecting users on low volume sites. The problem? Glue only works on high volume sites that your friends probably already use. Glue has promise but needs to work on all sites.
KPI has seen a big increase in the number of request coming in for SOA consulting. In the three weeks prior to the election, the rate of calls/e-mails coming in for SOA consulting services was down to 1-2/day. For the past two weeks, we have seen 6-7/day. There are a lot of new SOA projects getting underway, and that is good news for those of us in this industry.
We have previously blogged about government agencies needing to share information. The 300+ plus federal agencies each have their own records management program, all with varying degrees of maturity. Michael J. Kurtz, Assistant Archivist for Records Services, has a podcast up that is recommended listening. A multi-agency Records Management Service initiative, under the leadership of the National Archives and Records Administration, is in place to develop an approach to move to SOA. I am skeptical on getting 300+ agencies to agree and conform, but it's an undertaking that is way overdue nonetheless.
KPI has performed consulting gigs with the government and their environment is made for SOA. Many government agencies need to share data with each other. As you can probably assume, there are a lot of disparate systems in place currently making this a difficult task. Booz Allen Hamilton recommended the use of SOA to DOD and the Vet -- and the recommendation was accepted. They will use SOA for exchanging clinical data. Of course, the private health care industry already has the HL7 exchange format standard, which makes the health care industry perfect for SOA.
KPI strongly encourages clients to understand the principles of SOA, understand why SOA fails, and what makes SOA projects successful. Many potential clients simply want to dive right in. Reading case studies up front is a great way to mitigate the high risk of a failed SOA project (remember, up to 80% of SOA projects are still failing). Here is a case study from First American, a company which provides information to financial companies. FA faced the same issues facing many projects going towards SOA -- a lot of software running on different systems. FA built a consolidated data center, using virtualization and a mirrored site for DA. Most interestingly, they are looking at implementing Cisco's Service-Oriented Network Architecture (SONA). SONA is an architectural approach for designing advanced capabilities into the network infrastructure.
Money has an article online concerning the hidden costs of the Google cloud. The article focuses on the identity tangle that KPI has ran into -- identity management with different personal and business accounts. The article overstates the hidden costs, but it's worth noting that moving to the cloud will have unexpected productivity costs that management should consider in planning and budgeting.
Amazon continues to widen its gap in the SOA CloudScape with its launch of CloudFront. CloudFront essentially delivers content via a network of edge locations, promising low latency and high data transfer speeds. CloudFront is pay as you go (ie no sales negotiations or commitments). Woot uses CloudFront for image hosting. PlayFish also uses CloudFront to deliver games. This is one service which will tip the scales for many enterprises thinking about using the cloud. One thing to note, however: if you are transferring huge amounts of data, it will likely be more cost effective to use Akamai (which is essentially cloud computing if you think about it).
Tibco and OpenSpan have formed an unusual alliance. Tibo wants to integrate their SOA and BPM solutions into desktop applications, while requiring minimal changes into the desktop apps. Tibco's Tom Laffey (exec VP) states this "allows any application to become a service endpoint, event source or step in a BPM process." However, Laffey seems over-optimistic when he states "The OpenSpan technology helps our joint customers service and event enable any desktop application without changing the code and with nearly zero impact to that application." I assume he's talking about bottom-up web service design, in which you build a WSDL from existing code. However, bottom-up development has interoperability issues and code changes are frequent. When your vendor makes claims, be sure to ask questions. Vendor hype in SOA is decreasing but still abundant.
I wrote the other day about the current trend of power companies going SOA. BlueStar, an energy company in the DC area, is a classic use case for SOA. BlueStar buys electricity on the broader market and resells it to residential and business customers. BlueStar CTO Tom Keen says the key is to schedule delivery in an optimal way. This means managing the flow of information, such as billing and purchasing transactions, across many actors: power generators, transmission operators, local distribution companies, and customers. Read the entire use case in ComputerWorld.
Want to know why SOA projects fail? KPI was recently contacted for a "one week WSRR engagement." We were incredulous. WSRR comes with a default lifecycle which is too generic for legitimate enterprise use. WSRR configuration is not turnkey. You must configure a DB, configure LDAP, configure policies, etc. A proper WSRR set up takes months. The potential client, however, simply could not comprehend this. Obviously, we walked away. They had no commitment to a successful SOA project. They are doomed to fail.
A Joe McKendric survey in Septemeber and October found that all planned SOA systems were still a go. McKendrick said there is a lot of concern about ROI. Basically, companies are plenty nervous but still proceeding with SOA. A survery by KPI has similar results: planned SOA projects are not being discarded. Companies need to understand why the ROI risk exists for SOA projects. Generally, 80% of SOA projects have failed and that the root cause is almost always poor architecture and poor CTO. Most successful SOA projects have started with new people at top.
A few years ago, the auto industry moved to SOA. One company moved, then the rest followed. Now it's happening with the power industry. KPI has received several inquiries lately from power companies wishing to migrate to SOA. Interestingly, all of them plan to use WebSphere, Process Server, et al.
A new Network World article details how Hyperic lets IT managers monitor the cloud. The rapid development of supporting tools is a good and necessary step for widespread SOA adoption. The author states that enterprises are beginning to use cloud computing for overflow traffic. I agree, but more importantly, new enterprises are starting business with the cloud. If the enterprise looks like a winner, they intend to move onto their own servers for perceived savings.
Microsoft's Azure for Web Services, which I previously blogged about as their OS for cloud computing is live. SQL, .NET services, and a service bus are now available, but you will have to wait for Sharepoint and Dynamics CRM. I believe Sharepoint will be the most popular of their cloud computing apps. If you have questions, read their FAQ.
A study from CA finds that 43% of IT execs deem SOA security as the most critical issue to their SOA implementation. Surveyed execs reported experiencing an average of seven XML targeted attacks against externally facing SOA or Web services applications in the past year. SOA security is very immature and have their own set of security issues such as replay attacks. Over half of the respondents have delayed a Web Service due to security concerns. Execs are also calling for SOA security to be integrated with identity and access management systems.
Eclipse is ending several SOA projects due to lack of interest. Application Lifecycle Framework (ALF), started in 2005 to solve integration problems of application lifecycle management (ALM) for SOA developments, is gone. They tried to re-invent themsevles as a mashup provider, but for that, use sMash (projectzero) -- free from IBM. Eclipse will also end the STP Service Creation (STP-SC) sub-project designed to provide tool integration for SOA developers and the SOA System (STP-SOAS) set out to define developer APIs. As we often say, using open source projects is a dangerous game. Go with a prime vendor (Sonic, IBM, Oracle) who have enterprise quality products and support.
Amazon Web Services alum Jeff Lawson is launching twilio.com, a pay-per-use web service which will allow your web app to make and receive phone calls. Lawson forsees a retailer using it to create automated order status updates for a customer service line, for example. I do not see much value in this - who makes phone calls these days? It's all about SMS, Twitter, etc.
Gartner reports: "The number of organizations planning to adopt SOA for the first time decreased to 25 percent; it had been 53 percent in last year's survey. Also, the number of organizations with no plans to adopt SOA doubled from 7 percent in 2007 to 16 percent in 2008. This dramatic falloff has been happening since the beginning of 2008." This is attributed to the US economy. As I've written previously, however, Europe is much more progressive on the SOA technology front. They clearly see the technological and financial advantages of SOA and continue to think long term despite the short-term economic situation. This is reflected by KPI's industry experience and by readership stats on this web site. Gartner concurs, stating: "plans for (SOA) adoption vary widely by region. SOA adoption is nearly universal in Europe, moderate in North America, and lagging in Asia, Gartner said."
UPDATE: ZDNet looks at Force.com
Read this article to see how Force.com expands Salesforce's cloud footprint. SAP is falling even further behind Salesforce in getting CRM on the cloud. On an unrelated note, to see a cool Force.com/Zillow mashup, check this out: http://wiki.apexdevnet.com/index.php/Force.com_Zillow_Mashup
I have been working with a number of RESTful services lately. Unlike SOAP, some RESTful services do not have well defined responses. SOAP based services are (of course) defined in a WSDL, making the response easily parsible. The WSDL thus provides a formal contract between service provider and consumer. This is not the case with REST. RESTful service providers have not adopted WADL and I've seen everything from JavaScript to XML returned. In the worst case, RESTful providers are returning XML without an XSD. While RESTful providers generally provide APIs, this lack of a formal contract results in lost productivity. For example, there are some RESTful services in which I want to parse the response XML. Without a referenced XSD, I have lost productivity in researching APIs to figure out the response. This is unnecessary and something which must be improved in REST.
As you might be aware, Chrome was designed with SOA in mind. The threading architecture is designed to better handle connections to multiple services than other browsers. Chrome is a superior browser on the performance front, but it has one major flaw: zombie processes. Daily, up to 10 chrome.exe's appear on my task lisk despite all Chrome windows being closed. Sometimes the browser stops working correctly until the zombies are cleaned up. I have reproduced this on several computers -- hopefully this will be fixed by the upcoming patch.
Apple has silently updated MobileMe. I have learned it was a major rewrite of some of MobileMe's core code. Obviously, the MobileMe fiasco shows one of the major downsides and risks of cloud computing -- if the cloud goes down, a lot of people are affected. Until organizations can have local clouds which can sync data with a remote cloud, many potential customers are going to wait on the sidelines.
My company (KPI) has received several inquiries lately about architecting a new SOA project on a "2-3 month" contract. These inquiries are from respectable companies. It was hard for me to believe the naivity, so I took a couple of the initial calls -- and, in fact, they really think a new SOA system can be properly architected in 2 months. I do not like artificial timeframes. On these big systems, one has to understand the existing system and the requirements of the new system before beginning any architecture work. Architecture isn't the waterfall model -- every system will evolve as requirements change (and sometimes, that involves re-architecting portions of the system). The architect needs a presence during the entire lifecycle to ensure the development meets the architecture. As David Linthicum (and others) point out, having a good SOA architect is one of the keys to a successful project. Companies that try to be cheap and rent one for 2-3 months have a project that is doomed to fail.
Amazon is now offering virtual Windows as part of its EC2 service. Virtual machines running Windows Server 2003 and Microsoft SQL Server can be used on an hourly basis. Prices vary based on time and available capacity. Amazon's test period for renting Linux boxes is over and is now in the GA stage. Linux rentals cost 10c and hour with storage starting at 10c/gigabyte. Click here for the SLA.
David Linthicum makes some very broad SOA predictions in his latest column. He believes that PAAS is unstoppable and will leave many CIOs "scratching their head." Linthicum fails to see the security and privacy roadblocks to widespread PAAS and SAAS adoptions. But Linthicum is completely correct when he predicts that "Most failed SOA projects will be traced to unqualified SOA architects." KPI has found that projects primarily fail due to bad people. It all starts at the top and a bad SOA architect will completely kill your project. In fact, the most successful SOA projects (remember, 80% of SOA projects fail) start with a new CIO and a new SOA Architect. KPI highly recommends that organizations use consultants on new SOA initiatives. SOA consultants have a wide breadth of project knowledge and experience with all the vendor intricacies that you simply cannot learn in-house. Linthicum's other interesting predictions is that the "jig will be up for poor SOA governance solutions out there." I predict you will see a lot of merger and acquisition activity in the SOA governance world next year.
Being a graduate of the Carnegie Mellon/SEI Master of Software Engineering program, I am well versed in the CMM (Capability Maturity Model). Dr Dobb's has done a good job in modeling their Cloud Computing Adoption Model loosely around the CMM. The five levels:
   Level 1: Virtualization. virtualization technologies for seamless portability of applications and shared server infrastructure.
   Level 2: Cloud Experimentation. Virtualization is taken to a cloud model, either internally or externally
   Level 3: Cloud Foundations. Governance, controls, procedures, policies, and best practices begin to form around the development and deployment of cloud applications.
   Level 4: Cloud Advancement. Governance foundations allow organizations to scale up the volume of cloud applications through broad-based deployments in the cloud.
   Level 5: Cloud Actualization. Dynamic workload balancing across multiple utility clouds.
It is hard to improve what you cannot measure. These levels will help organizations measure the maturity of their cloud.
IBM has traditionally marketed their products at large conferences and direct to the C-level. But with SOA, they are trying something new. IBM is in the midst of a 10 day, 100 city "grass roots" tour. One-third of these cities are in emerging economies, with each event customized to the local market. In general, both an IBM rep and a local customer will present. This campaign is ingenious and something that will be hard for Oracle to compete with. While Oracle continues to beef up its SOA infrastructure by buying companies, they simply cannot compete with IBM's vast service network. And now, IBM has found a nice way to use this network to their advantage in gaining SOA market.
As you may know, Amazon's Web Services started for in-house use. eBay might be following that example in exposing it's huge data warehouse for internal business analytics as a web service. eBay has 5 petabyte of data and adds 50TB every day. eBay has used this data to create virtual data marts used by 5,000 business analysts.
A survey by David Linthicum showed 75% of respondents believe that a down economy will help SOA. KPI does not see this happening because of the high initial investment. We feel that company's IT budgets will be spent mostly on keeping the door open the next 6-12 months, and then you will start to see more investments in a company's future technologies. SOA does result in a competitive advantage when done correctly, but too many SOA projects are still failing.
Starting November 1, S3 will be cheaper. Under the new model, Amazon will still charge 15 cents per gigabyte of storage per month. But, after 50 terabytes, the price falls further (see new pricing list). S3 is up to 70,000 storage, retrieval and deletion requests per second. If you see s3.amazonaws.com on the bottom of your browser, the data is getting loaded from S3.
KPI's Gary Patalsky highly discourages customers from using open source SOA products in enterprise environments (go with Oracle or IBM - you will need their support). But if you are already a JBoss user, JBoss Enterprise SOA Platform 4.3 could possibly be an option. V 4.3 has new ESB (new gateway listeners, a declarative security model, improved web services integration and additional scripting languages) and Rules (Stateful Rules services, decision tables and Rule agent support) features.
Steve Ballmer says that a Windows Cloud OS is four weeks away. Ballmer notes that SharePoint and Exchange are already in the cloud. As part of this push into the cloud, Microsoft will put more emphasis on .Net, Silverlight, Active Directory and SQL Server. Let's hope the Cloud is better than Vista.
HP Systinet 3.00 has been released. New to version 3.0, multiple users within an organization can discover and reuse services. With the upgrade, customers can also automate service lifecycle policy compliance by capturing best practices to achieve SOA objectives (via separate product Quality Service Center).
Need to set up a database? Don't want to set up a server with MySQL? Want a quick enterprise level DB? Try Oracle on the Amazon cloud. We've written the news but now it's time for a review. KPI's review: It works! It's a breeze to set up. And it allows you to focus on your core business value, not in wasting time setting up servers and databases.
Google has opened up shop in Washington (Reston, VA to be precise). They have fit in nicely so far, building a nondescript building to go along with all the other nondescript buildings that plague Reston. The feds still does not get the cloud, says Mike Bradshaw: "Sometimes they'll look at us and say, 'But what do you actually sell?'". In all honestly, I've worked with many government agencies and most of them get SOA. Of course, in SOA, you are buying something. In any event, federal agencies should NOT be using Google's cloud. They should build their own cloud, with their own security. Do you trust Google with your private data? I surely do not.
An interesting topic which I had not considered is the effect of the cloud on today's kids. The Happy. Healthy. Life. blog tackles this fascinating issue. Many parents have age limits on social networking, but at what cost: sites which use the cloud (Facebook, Twitter, Whrrl, etc) bring people close to together. But there are still privacy and security issues involved, two areas that are still immature in the Cloud. As the blog states, a happy medium of cloud usage is the key.
Joe McKendrick writes that now may be the right time for all the SOA startups. This is truly a great time to start a startup. The problem? Funding. Try getting VC funding right now on a technology which has received a lot of press for its 80% failure rate. The financial industry, which would benefit greatly from SOA with all the current mergers, have not been big adopters. Instead of startups for SOA technology, I believe we will see more startups using SOA technology. By using cloud computing (such as Amazon) to provide storage, data storage, etc. on a much cheaper basis than having to build out that yourself.
Sometimes I wonder if Larry Ellison knows that Oracle has been seeding the cloud. Ellison said Oracle will not invest in cloud computing centers, which is not Oracle's expertise. However, there are lots of companies making money on the cloud. Amazon for one. And we know Oracle delivered AMI's for one reason -- $.
Vendors continue to rush to unveil new SOA products and add-ons, obviously a positive sign for the technology. Software AG unveiled webMethods SOA Suite which is the webMethods "ESB". Whether this is a true ESB, or rebranded old technology as I suspect, remains to be seen. Software AG has been partnering with Layer 7 to expand their SOA exposure. But as KPI advises, go with Oracle or IBM if you are looking for complete SOA solutions.
Mule Xpack for Intel(R) XML Software Suite is a set of Mule extensions that significantly improve XML processing performance for SOA deployments. XML processing performance, as we know, needs a lot of improvement. There needs to be an industry XML binary standard. But until then, the likes of StAX and Intel XML Software Suite will need to do. Intel's XML Software Suite provides thread safe and efficient memory utilization, scalable stream-to-stream processing, and large XML file processing capabilities. XPack can be used for XML parsing, XSLT transformations, and XPath evaluations. As always, however, KPI recommends that you do not use open source SOA products for your enterprise.
Sure, it has been available in a feature pack, but IBM's feature packs do not have the same quality as a GA (in general). The adoption of WAS7 will further push organizations into the SOA World. The current version of WAS has become badly outdated, relying on the extremely slow JAX-RPC. And with so many "IBM shops" out there...
Oracle is entering the cloud. They will deliver a set of free Amazon Machine Images (AMIs), enabling customers to deploy Oracle solutions on EC2. Using these AMIs, new virtual machines can be provisioned with Oracle Database 11g, Fusion, and Oracle Enterprise Linux. Oracle is also introducing a secure Cloud-based backup solution: Oracle Secure Backup Cloud Module, based on their tape backup management software, to enable customers to use S3 as their database backup destination. The Oracle Secure Backup Cloud Module also enables encrypted data backups to help ensure complete privacy in the Cloud environment. It's fully integrated with Oracle Recovery Manager and Oracle Enterprise Manager, providing users with familiar interfaces for Cloud-based backups. Secure, encrypted backups is an absolute necessity when running on a public cloud.
Parasoft and AmberPoint announced closed-loop integration between their SOA quality and runtime governance solutions. The integrated Parasoft-AmberPoint solution promises to automatically emulate services based on real-world historical data collected from the runtime environment. The advantages are obvious -- testing complex distributed SOA systems is one of the main challenges of SOA.
Amazon, the undisputed king of SOA, will begin a pay-as-you-go content delivery service. How simple has Amazon made it? You start by storing the original version of your objects in Amazon S3 to ensure they are publicly readable. Next, a simple API call registers your bucket with the new content delivery service. This API call returns a new domain name for you to include in your Web pages or application. When clients request an object using this domain name, they will be automatically routed to the nearest edge location for high performance delivery of your content.
Here are the slides to Linthicum's speech "SOA from the combat zone" at the InfoWorld SOA conference in NYC. KPI finds that Linthicum's basic principles are correct but often ignored by companies. One instance where Linthicum is simply wrong: when he states 'do not let consultants or vendors drive the project.' Your SOA project will have a much better chance of success if you let IBM Global Services or Oracle drive. Why? Because they have the experience of many SOA projects already behind them. Likewise, an experienced SOA consultant can bring a wealth of knowledge from different projects and different vendors. The last thing you want to do is to use in-house developers. In fact, the most successful SOA projects are ones that start with a change at the C-level, followed by imported SOA architects and developers to augment and teach the in-house staff.
Morph Labs has launched Morph AppClouds, a PAAS (platform as a service) built using Amazon Amazon Web Services (Amazon EC2 for virtual servers, as Amazon S3 for storage). Through the Morph Labs Web interface, users choose the components they require for their specific environment using drag and drop. With one click deployment, the cloud is provisioned, configured and dozens of technologies are enabled to form a comprehensive end to end environment for the deployment, delivery and management of Web applications. Cloud computing is progressing at a fast rate -- from both vendors and business. I would like to see more attention paid to security and the ability to access the online data offline if the cloud goes offline (that is being worked on) as these are the two main barriers of entry for many companies.
The Roosevelt Hotel hosts the InfoWorld SOA Executive Forum September 16th and 17th in NYC. David Linthicum presents "SOA in the Combat Zone: How to Succeed with SOA the FIrst Time, and Never Make a Mistake" on the 16th at 8:45am. Another good talk will be on 16th as well in the afternoon: "SOA Security: Why the Old Rules Won't Work". TheSOABlog will be following this conference -- check here for updates.
EMC, IBM, and Microsoft announced a joint specification on interoperable Web Services for enterprise content management. The companies intend to submit the Content Management Interoperability Services specification to OASIS (Organization for the Advancement of Structured Information Standards) for advancement through its rigorous standards development process. Currently, companies waste a lot of money in productivity on content management. There are several advantages (1) An open specification will result in a higher knowledge based as developers are tied to a vendor specific implementation (2) Organizations with multiple CMS systems will be able to integrate the data more easily (3) Web Services will be decoupled from the CMS database.
As some readers may know, I was on the initial development team of the IBM WebSphere product at Transarc/IBM. I have worked with the IBM Hursley lab and other non-IBM UK based companies. There is a great divide between America and the UK on software process. The UK embraces it, while American managers, in general, tend to make penny fool decisions on the matter (as additional evidence, the UK spends more time reading TheSOABlog than the US!). As the leader of KPI, I stress to our clients that a good process and SOA governance is critical in large SOA systems. Without it, your SOA implementation will fall into the 80% failure bucket. Imagine the scenario where you need to evolve your service version to a new version. Remember that in SOA, a service provider should be decopuled from a service consumer. How does a service provider advertise it's expiration date, if any? How does it advertise a new version? How is service created or retired? Which brings us back to the UK, where the Hursley lab has been instrumental in WSRR, IBM's registry and governance tool. After an organization has the proper mindset on process and governance, they need to pick a tool. KPI recommends WSRR (there are other tools we like also) as it gives business the opportunity to create their own governance model that supports the process and governance needed for a large scale SOA deployment. If you are using the IBM stack, it is also becoming increasingly integrated into DataPower and WebSphere.
KPI always advises our customers to be on the lookout for SOA vendor spin, especially vendors putting the SOA label on old technology. This vendor-written article on the benefits of open source SOA is just one example. The spin begins in the second paragraph: "Open source has become a staple of enterprise-class IT as concerns about stability, security and support fall away." These concerns are, if anything, growing -- especially in the area of security. Fortify has shown that some of the most popular open source systems are exposing businesses and users to "significant and unnecessary business risk." The article then goes on to advocate open source for your ESB and governance, calling it a "good and affordable option." Both claims are ridiculous. In high throughput enterprise level environments, you need an enterprise quality ESB and enterprise level support. Open source might be free, but if you use it for an emerging technology such as SOA, you will pay more in the long run. KPI has seen it happen over and over.
UPDATE: if you still trust Google's cloud, read this

With cloud computing gaining momentum, the need to protect your private data is more important than ever. Did you know the Feds are using Facebook, where you can easily create a fake profile for someone, to help decide who poses a threat to national security? With Facebook, Whrrl, and Twitter becoming tightly integrated into the cloud -- the need to protect your privacy increases. This brings us to who wants to become the biggest cloud of them all: Google. The world's largest search engine stores private data - it's one of the reasons they target ads so well. Privacy and security lags significantly behind other components of SOA and the cloud. KPI recommends to our customers some basic steps to take: (1) always browse in 'private' mode, a feature in Safari and the new Microsoft browser (2) aggressively search for false Wikipedia, Facebook, etc. entries and (3) use high privacy/security settings in applications and browsers. You would be shocked to find out how even these little things are not done.
Active Endpoints released version 6.0 today. ActiveVOS allows you the build service orchestration and BPM applications using open standards. It was selected by the FBI for orchestration as part of a 10 year contract awarded to Lockheed Martin. ActiveVOS is also used by Toyota (Toyota is a big believer in SOA).
IBM correctly summarizes one of the best IBM best practices in a recent whitepaper: "To achieve a successful (SOA) deployment, a core architecture leadership team must first be established to ensure consistency of efforts and direct the vision of the architecture." IBM's five best practices: (1) deploy architecture with a vision for the future. KPI's experience is creating services and WSDL's with the future in mind is essential (i.e. use the extension pattern as Amazon does). (2) Foresee linkages from IT to your business processes. (3) Create an organizational structure to support SOA. KPI finds that SOA must be supported and the organization structured to support SOA (such as governance) or else your SOA deployment will fail. (4) Build a scalable infrastructure (5) Enable operational visibility on governance and service management. Again, this starts at the C-level.
Motley Fool has a spot-on article regarding the business and investing side of cloud computing. First, Motley states the recent outages from Google, Apple (.me), and Amazon (two hour cloud outage) shows businesses should not be running their business on the Internet quite yet. That is obvious. Much more importantly, Motley Fool also calls you "crazy" if you're not betting on the cloud as an investor. The reason? Google, Apple, and Microsoft are all working to offer offline access to their online data. Google is working on this with Gears, Adobe has the AIR runtime, and Microsoft, of course, has Silverlight.
The technology behind Google's new browser is a step forward for WOA. The thread and memory management was built in such a way to better support WOA and web services, unlike the current browsers which rely on legacy code. FireFox, for example, is well known to freeze when too many tabs are open with Flash components. Google's intent can be seen on the home page, where a number of independent web pages can be seen at a glance. SOA has been receiving a lot of negative press for its 20% success rate. While usually due to overreaching implementations and bad people, Google (which has lagged considerable on web services behind Amazon) just gave a WOA a big boost. This will help WOA by forcing Microsoft to get more involved in WOA/SOA.
On the heels of Progress buying MindReef and IONA, Oracle has acquired ClearApp. ClearApp monitors SOA performance and tracks runtime environment changes to ease the burden of finding application and performance problems in complex SOA environments. ClearApp also supports WebSphere. In some SOA environments, "IBM shops" using WebSphere might now use three components from Oracle: Tangosol, ClearApp, and DB. Oracle is clearly positioning themselves to have a homegenous solution in order to siphon market share from IBM. Good move, Oracle. Your turn, IBM.
SOA has their own set of security vulnerabilities. Besides vulnerabilities such as weak encryption that have been faced before, new ones such as replay attacks must be considered. Fortify 360 is one tool to look into for help with these issues. Most importantly, think about security from the start. Your performance metrics might go from acceptable to not once you go from weak to strong encryption, for example. And that could trigger a vicious cycle where you need different hardware or tweaked architecture.
SAP will have new enhancements by December geared towards SOA, reports Computer World. The "more consumable" features of SOA will have service definitions.
I have taken a lot of shots at Microsoft for their general cluelessness in the SOA space. An article in CIO confirms what we already believe to be true: Ballmer is the key to SOA at Microsoft. The article is worth a read as it looks into how the Ballmer/Gates philosophies affect SOA at Microsoft.
From SQL server 2008: 'In SQL Server 2008, SQL Server native SOAP has been deprecated and will be removed in a future SQL Server release ... Avoid use of SQL server native SOAP in new development work, and plan to modify applications that currently use it." SOAP doesn't belong in the DB but applications looking to upgrade should be aware of this issue.
KPI highly encourages its clients to adopt SOA governance. In fact, we believe SOA implementations without governance will fail long term. Springboard Research has published a report showing that SOA investments will increase 60-90% in Asia over the next couple years. This will force SOA vendors to invest more in their governance tools (investment by vendors is lacking currently). 40% of SOA adopters are now making governance a requirement, a percentage we find to low. Start governance from the beginning of your project -- do not wait until the end.
VOXWare sells SOA based software that processes orders using speech recognition. By the few accounts out there, it has been a success. But their customers aren't talking -- probably because SOA gives them a huge competitive advantage. Even CIO is having trouble getting details. This is what KPI has found on SOA implementations. They have high upfront costs and a learning curve, but the end result of a good implementation is a large competitive advantage (due to the flexibility, especially in the event of an acquisition).
Amazon, as you might know, has EC2 for on-demand computing and S3 for on-demand storage.   A huge problem was that EC2 storage was non-persistent, forcing you to use S3 for storage.  There were workarounds such as RightScale and PersistentFS.    That has changed with Amazon's introduction of EBS (Elastic Block Store).  Volumes from 1 GB to 1 TB can be mounted -- read this post on GeekZone for more details.
KPI has found many customers looking to use SOA as a wrapper to their legacy systems.  But would you use SOA as a startup, as suggested by Mike Davis in CIO?   I have been the earliest memeber of a startup and know that startups have to focus on their core technology.   I found myself scratching my head while reading this article; this sentence is particularly nonsensical: "But when it comes time to deliver reliable production versions, startups should consider implementing a light weight SOA to achieve the long term benefits of reuse, agility, and flexibility".  You don't need SOA to deliver reliable production versions.
Services need identities too is the major point made by IBM's Raj Nagaratnam on SOA security.   Few organizations understand that SOA is a more complicated technology than a straight J2EE application.  Given that services are loosely coupled, security becomes more complicated.   Raj states people are thinking about individual services how to reuse them but they’re moving to where it’s a model where multiple services could be composed to traditionally security measures that oriented towards a single application or a service. But then, we compose these multiple technology services into business services and policies need to be managed at a very high level and not just at technology like a web service level but holistic business service level." I encourage you to read the full article.
Joe McKendrick writes that SOA governance must improve in ZDNet. Interestingly, McKendrick states that some do not like the word governance itself.  But SOA really does need governance, especially when it comes to versioning, SLA, etc.  Without governance, your SOA can quickly become an unnavigable mess.   KPI has experience with a number of SOA governance tools, including AmberPoint, WSRR, Actional, and Systinet.   None complete the total solution, but AmberPoint seems to be the most dedicated to governance.
The Peter's Principle generally applies to managers who focus on a knowledge of specific tools or framework, instead of focusing on a good person with applicable skills.   It is a major error made consistently in IT.   David Linthicum presents a real world example that KPI Global Technologies finds all the time:  a company or recruiter calls KPI and asks, for example: "does your organization have at least one year experience in WSRR?"   What they should be asking is: "How much experience do you have in SOA governance."  The tool is secondary.   SOA governance is about people and process.  Many organizations fail to get this simple point -- and is one reason why so many IT projects fail.
Paul O'Connor gives JaxView a good review in SOA World, stating it's a good tool for budget conscious organizations looking for a web service monitoring solution.  The UI is basic and will not give you the impressive visuals that Mercury can give, for example.  But JaxView is straight forward to install and will give you the metrics you need, in addition to an API  to develop your own active monitors.
Progress is on a roll.  They acquired Mindreef, an SOA QA and validation company, two weeks ago.  Now they are acquiring IONA, a leader in CORBA technology for the last 15 years.  This move helps Sonic win customers looking to integrate their C++ based systems into the SOA world.  Progress now has Actional, Mindreef, and IONA.   Sonic has lost a huge amount of market share to IBM and Oracle, but these acquisitions strengthen their position in the market.  
CIO has a nice article on the SOA sales pitch from vendors.  Vendors are being more realistic (and honest) in their sales pitches.  The article also details a strategy for buyers: "There are other price negotiating advantages that can be leveraged by savvy buyers. "Start with the vendors you already work with," advises Kennedy. "You might not have to buy as much to make SOA happen for you. Take, for example, enterprise service buses: Tell your vendors, 'I have problems making your widget work with my SOA plan, so get in here and help me fix it.'"
An article by Ty Anderson of CIO.com compares SOA to The Total Gym.  The crucial point -- one which must be understood by organizations thinking about SOA -- is when Anderson states"Adopting SOA is lifestyle change that has to become integrated into all aspects of your culture, including business processes, software requirements development, software architecture, software development lifecycle, project management, and hardware and software purchases."
NetworkWorld has a nice article on starting a SOA initiative.  NetworkWorld's last piece of advice might be the most important: do not try the "big bang" approach.  It is not recommended by vendors and will cost you a lot of money.  In addition, the importantce of a good SOA architect should be emphasized.  KPI highly recommends organizations to spend the $150/hr on a good SOA architect.   Given the amount of money companies are losing on SOA, a good SOA architect at $150/hr is cheap (and has a much higher ROI than spending $150/hr on two SOA folks).
Page 2 of this article in CIO gives the state of the union for leading SOA vendors (and also briefly discusses handling vendor consolidation).
We have heard about Microsoft's BizTalk Services which brings orchestration into the mix.  Microsoft is lagging far behind the competition on SOA, but they have recently been taking more initiative on the SOA front.
A failure in a SOA system at Heathrow resulted in a $3.2 mil one week loss.   Public gaffes such as this bring testing to the forefront.  How do you test SOA orchestration, load, and performance?  An SOA system could have layers of transactions, making testing complicated.  And as engineers know, testing is usually substandard on the easiest of systems.   KPI suggests that the testing begin at the same time of development.  Do not, under any circumstance, leave it to the end.   Computerworld has a SOA Testing primer online.
Amazon is an industry leader in Web Services.  Check out Animoto.  It analyzes your music and creates a custom video.
Yahoo! is taking their search SOA according to this article.  This is a good move by Yahoo!  Amazon's web services have been hugely successful.  From Yahoo!'s business perspective, anything to retain or increase market share is a good thing.
Traversing the SOA jargon can be difficult. Beware of vendors are sticking the SOA label on old products.   Companies are beginning to tune in to this trick, however.  Read this article in PC World about vendor politics. 
Should we call them Big Blue Cloud?  I am an original developer of IBM WebSphere, back to the beta days, when there were about 20 developers total on the project.  Turn the clock ahead 11 years and IBM is now building a building a $360 MIL cloud computing center in a rehabbed building in RTP. 200 full time researchers will be on the project. With DataPower, Process Server, WSRR, and initiatives such as this, IBM should remain on the forefront of SOA.
Linthicum offers nine steps on when to hit the SOA reset button.  He correctly notes the pattens of success involve: an accepting culture, smart people, good leadership, and enough resources to do the job.  His reset methodology is simplistic.  In the case of an unaccepting culture, a good SOA implementation will need a change at the C-level and elsewhere for any chance of success.  This is extremely important on projects that use an ESB to bridge to a mainframe.  As an example, take a mainframe company attempting to use Sonic as their ESB to the mainframe world.   KPI has turned down several such initatives because we deemed that they were doomed to fail:  poor leadership, poor understanding of ESB, and engineers stuck in mainframe mode.
A lot of press currently about the failure rate of SOA.  Here's another article, asking if SOA = "You're Fired" to CIOs.  KPI stresses to our customers that SOA can give you a competitive advantage, however, you that you will bleed money if you do not understand that SOA is a paradigm shift for most organizations.  If you are looking for a model to follow, look no further than Amazon.
Probably half of IT projects fail. This study shows 80% of SOA projects failing.  But the article does give common threads among the initiatives that have succeeded.  KPI's experience is that C-level buy-in and a cultural shift are necessary for any SOA initiative to succeed (and that often comes with a change of CTO or a visionary one already in place).
Article on ZDNet about SOA increasing the cost of defects. McKendrick is completely wrong when he says the purpose of SOA is to "simplify things".  The purposes of SOA are integration, interoperability, and a competitive advantage in acquisitions.  SOA, of course, introduces new methodologies and new products into the organization.  This will in turn increase the number of errors.  Process is essential to SOA -- the earlier you find a defect, the cheaper it costs to fix.
Analysts and non-technical tech writers are using the term "cloud computing" with increasing frequency.   Cloud computing means too many things.  In most cases, analysts simply are clueless when throwing out the term.   Here is an article that attempts to define three types of "cloud computing".
At Network World, a panel conclusively agreed that SOA efforts are failing due to lack of process.  SOA is a new way of developing software and your process must reflect that.  Otherwise, your project's cost  will rise uncontrollably due to re-engineering efforts, something KPI has seen.
There is an article about Mule Galaxy Enterprise, the first open source SOA with integrated registry and repository.   KPI advises companies to stay away from any open source SOA solution.   Quite simply, SOA initiatives need good vendor support.
Dave Rosenberg is a master of the obvious when he states that Microsoft is missing the boat on SOA.   IBM comes closest to a single-vendor solution (WSRR, DataPower, WebSphere, Process Server, MQ) while Oracle is closing in with its many acquisitions (BEA, Tangosol).  For new SOA initiatives, KPI still recommends IBM.