Enterprise integration requirements are rapidly shifting to accommodate such trends as cloud computing, mobile devices’ explosion, and increased demand for extended enterprise business processes.
Application-to-application integration inside an enterprise’s four walls is well understood, but very quickly the demands placed on integration are spanning multiple enterprises, multiple types of applications, and varieties of service providers. As a result, Software as a Service (SaaS) and cloud computing are joining with legacy systems to form new and varied hybrid models that require whole new sets of integration needs and challenges.
Once these newer breeds of integrations are set up, can the old, brittle management and upkeep of them suffice — or will agility and rapid upgrades and innovations require new tools to make integration a lifecycle function with ongoing management and more automated governance?
In the latest BriefingsDirect enterprise IT discussion, the panel examines how open source integration projects like Apache Camel and lightweight integration implementations and graphical tools are making developers and architects more agile. At the same time, these open source approaches are proving less vulnerable to the complexity, fragility, and cost that often plague aging commercial middleware integration products.
Dana Gardner, principal analyst at Interarbor Solutions, recently sat down with Rob Davies, chief technology officer at FuseSource; and Debbie Moynihan, vice president of marketing at FuseSource, to examine the need for innovative, new, open and agile integration capabilities.
Listen to the podcast (35:10 minutes).
Here are some excerpts:
Dana Gardner: The need for integration is increasing. The things that need to be integrated are increasing, rapidly. Open source is well established. When you put these factors together this perhaps spells an historic shift. Has the ability to integrate openly become an essential ingredient of businesses?
Rob Davies: Sometimes, it’s difficult to see things happening like that, if you’re actually right inside in the middle of it. We probably are at that shift right now.
We’ve talked about cloud environment. Also there’s social networking, SaaS, and mobile devices, and you need to link all those together. It’s coming to the point where organizations won’t have a choice other than to use open source as a way to try to keep up with a pace of change.
We’re probably at a point now where we’re going to see that the traditional model of providing software is going to dwindle over time, probably pretty rapidly, as organizations realize that they need the flexibility and the ability to change what they’re doing very quickly.
You have to start thinking about how you’re going to future-proof your applications right from the beginning to adapt to changes in their environments. You have to architect in how you’re going to integrate and future-proof your applications, because it does get more costly if you do it as an afterthought.
Gardner: Many of the SaaS providers are doing multitenancy and providing applications as services on demand at a very attractive and aggressive price point. They’re leveraging open source on the back end, I have to imagine. Do you have any insight into what the service providers themselves are building with?
Davies: Most applications now — in particular in the cloud — are using open source at the back end. We can’t give you any specific details of vendors that are doing that, but I know they’re using open source projects, and not just the SaaS vendors, but some of the other existing product vendors use open source as well to enable their products.
We certainly see open source as definitely mainstream now, and we’ve seen it has been the first choice that people use for building any kind of application or service they’re providing. It’s more a case of people asking the questions now of not should we be using open source, but why shouldn’t we use open source? It’s starting to become a first choice for people to go to.
Gardner: Debbie, why do people need to rethink integration?
Debbie Moynihan: The business models are changing and people are being asked to do more with less. Teams and applications are more distributed than ever.
There are a lot of new technologies coming out that people are struggling to learn, and figuring out how to incorporate them into their infrastructure: cloud, mobile, the explosion of the huge amounts of data that enterprises are trying to understand and make sense out of. Not to mention the social media technologies that people are being asked about and wondering how to incorporate into their enterprise infrastructure.
There are a lot of different skills that people are looking to have that they’ve never been asked to have before. More and more people are being asked to perform IT tasks. It isn’t just highly skilled developers, but also business analysts and people who have never done integration before are being asked to do integration activities.
They’re not sure how to keep up with all of these changes. Costs are a problem because essentially everyone has the same or smaller budget going forward and a lot of people have fewer people to do what they’ve been doing before.
At FuseSource, we’ve seen a lot people looking more and more to open source to solve some of these problems. … There’s a lot of flexibility. When the environment changes and new technologies come out, you need to integrate new things into your environment.
The community people, when they see a problem or new technology, just make it happen. They can add, expand, and modify what’s involved in the various open source integration projects without the overhead and bureaucracy of some of the traditional software development environments.
Gardner: In the past, when we had a shift in computing, we’d bring in a new set of applications, we’d update our platforms, and then think about integrating them. It was a sequential process, and it could take three to five years to go through something like that.
We don’t really have that luxury anymore. Now things are happening in a simultaneous fashion. So integration really can’t be an afterthought, but needs to be part-and-parcel with how you go about designing and implementing your applications.
Doesn’t open source, in a sense, allow for a compression of the time that we’ve traditionally taken with commercial products?
Moynihan: Absolutely. open source is a componentized, lightweight approach. As people develop their applications, they develop them in such a way that they can be broken apart in new and different ways down the road, and it’s very transparent. It makes it easier over time to further integrate what you’ve built and to make changes as you need.
Gardner: One of the other aspects of this that I’m seeing in the market is that more people need to take part in integration. It can’t just go through a bottleneck of “beard-and-sneaker guys” in the back room who can do coding. Integration needs to be part of process innovation. That means we need to elevate it out to a wider group of individuals, maybe as many as possible that are on the front lines of process innovation and analysis.
What’s being done about the integration that we’ve been describing to make it more, well … applicable?
Moynihan: On April 11, we announced the general availability of a new graphical tooling for Apache Camel. [Users can download a trial version of the plug-in, which includes some of the functionality of the fully paid version found on the subscription-based Fuse Mediation Router.]
The addition of graphical tooling makes it easier for more people to do integration development. They don’t have to write code. They can use a drag-and-drop environment to select the integration patterns that they want to implement, and the software will implement them. They can test them and deploy them into production as well.
The addition of tooling is going to help broaden how many people can do integration, and we’re real excited. We’ve been doing a beta program since the end of January with over 500 participants. Rob mentioned the breadth of all the components and how hot Apache Camel has been. We’re not surprised that more and more people want to use it. So the idea of having tooling on top of it is really attractive to users.
Gardner: So, what’s the name and where do you go to find out more about them?
Moynihan: The Fuse IDE for Camel is the name. It plugs into an Eclipse environment and you can get it at fusesource.com.
Gardner: You know it strikes me that when we begin to talk about integration that I’d mentioned service-oriented architecture (SOA), but that was sort of yesterday’s buzzword. We’re now into cloud, hybrid and mobile. But from an architectural perspective, you can’t really scale and leverage these open components without that proper underpinning, typically an enterprise-service-bus (ESB) architecture.
Rob, help me understand why doing this correctly from an architecture — not just an open source — perspective is really important as well.
Davies: You hit the core things about the SOA and the ESB architectures. We see where people are using, in particular, Apache Camel and some of our other open source projects. They want flexibility there. So, they want to leverage a service bus, put things on, expose them as service, and expose them over the service bus, which uses different transports to enable that bus, be that messaging, HTTP, or whatever other means you want to use.
At the same time, you also want to have the flexibility now to do it in application integration. You want to have that flexibility for some services and you very much need that enterprise service bus in place. But for other cases, you want to be able to do that more locally, where the integration points are.
The approach that we have is that we enable you to do both, because you can embed Apache Camel inside an application server, if you want it inside your application itself. If you want to use it in a more traditional sense, you can deploy it into ServiceMix. You can define your apps easily, deploy them into ServiceMix, and use it to manage the container.
Having that flexibility as well means that you can have the right architecture for your particular solution. If you look at how people would do the integration before, they’d have to get an ESB, and that would force the whole architecture of how they do things. When you’ve got more flexibility, it means that you can make the right architecture choices that you need, and you’re not constrained to one particular style of integration.
Gardner: I’m facing a lot of questions more recently about how to architecturally cross the domains that we’ve mentioned — SaaS, cloud, on-premises, traditional architecture, and private cloud architecture.
Does the service-bus approach and the open source approach also give us some sort of a path or vision for how to go about this?
Davies: Having open source enables you to have the insight into how the integration application works.
If you just look back just a couple of years, when people were starting to use the cloud, they weren’t even thinking about having hybrid clouds. Now, we’re seeing more and more people, more of our customers, looking to hybrid clouds and have a private cloud for applications.
When they need the capacity, obviously they can get that capacity in a public cloud. But, to have all those PCs working together seamlessly, they need the agility that you get from an integration solution that can be deployed on a public cloud, locally, or a combination of both. That’s something that you can only get from software that has evolved at the same pace as the demands of the environment.
You can only really get that speed of innovation to keep up with the way the environment is changing by choosing open source, because the open source community itself is driving the projects to keep up with the demands.
So, you have to try to move outside of a traditional release cycle that you would get from a traditional product company. You don’t really have any other alternatives, if you want to keep up, than to look at open source projects, the Apache ones in particular.
Apache projects certainly hit the right notes in that you’ve got both very business-friendly license from the Apache license and very active communities, and you’ve got diversity in that community. You know these projects are going to live beyond the lifetime of particular individuals on the projects.
You also have the benefit of having companies like FuseSource, which created the projects in the first place, and who are there and able to provide support and consultancy if you need it. You get the best of having a dynamic community, a dynamic project, and you also get the security of having professional company to back it up.
Gardner: How rapidly are the iterations within the Apache project, within Camel in particular, happening? How rapidly is innovation taking place?
Davies: It’s happening at a very fast pace. When we do release these out of Apache, it’s typically every three months, but in that three-month period there could be other components that have gone into the Apache Camel Framework. Because it’s open source, people can actually look about, release their own components into an open source environment, or develop them separately without necessarily releasing to Apache, just to get the functionality out.
That pace of change is very fast and it’s near real-time. When the need comes up, within a few days or a week, you would probably find someone who has already written that integration component that you need and it’s available. … If you’ve got an open source framework, you can actually have an insight into how the project works.
After we launched Apache Camel at the Apache Software Foundation, we provided a number of default integration components for Camel. But as soon as they got out there and the community started to use them and saw the benefits of using them, we saw no end of contributions. People contributed adapters to weird and wonderful systems, and contributed them right back into the Apache project.
We know from our customers that they’ve got specific needs. They’ve got legacy applications. Because we’ve gone to the effort of making sure that it’s very easy to add a new component into Apache Camel, it’s very straightforward for someone to add in extra functionality.
For example, if you want to write a component for legacy mainframe application, you could very easily do in a matter of hours. The old approach would take you weeks, months, maybe even years, especially if you don’t have access to the source code. So, you’ve got that added flexibility.
The fact that it’s an open source project at Apache means you can get feedback instantly, if you’ve got issues and problems. Of course, if you want professional help, there’s FuseSource as well. We have our own community at fusesource.com. So, all these things combined means that you have more flexibility and a much more agile way of doing integration.
Loading ...
Great article. Especially your referral to graphical toolsets (build functionality with drag-and-drop) and the trend to use less or no code at all. I’m a big advocate of this trend which empowers me as a ‘non-technical person’ to be closely involved in application development.
An interesting dimension to SaaS and cloud computing is the concept of PaaS (platform-as-a-Service) which would perfectly fit this story. Being an employee of a software firm (under full disclosure – Mendix) that delivers a visual modeling platform I believe that SaaS is best complemented with a PaaS cloud layer. Not sure if your toolkit fits this category but our CTO wrote a very interesting article about it; for your interest please refer
to: http://www.theenterprisearchitect.eu/archive/2010/04/27/multi-tenancy-and-model-driven-engineering-necessary-assets-of-a-platform-as-a-service