This Thursday, September 12 at 11 AM EDT, the GWC will host a webcast: Meet the new “mini” Power-based PureApplication System and latest IBM Patterns.
Planned pattern topics for this webcast are new IBM patterns covering the newer version of our BPM solution, as well as Sharepoint and MySQL solutions.
This session is free to attend. You can register on the GWC website.
If you’re not able to attend the session live, you can listen to the recording. To access the session recordings, go to the Recent Archives tab on the Webcast page.
This post was written by John Hawkins, a Principal Consultant at Icon Solutions.
I’ve always found the complexity of the plugin model somewhat overbearing, so I figured it would be good for me (and hopefully others) to explain the aspect that comes up time-and-again: how does the model that the user draws on the canvas actually get translated into the underlying topology model for deployment to the hardware?
The plugins that you can see on the canvas are logically split into three main elements: the model, the packages and the transformers:
The model contains what literally appears on the canvas of the PureApplication Pattern: the modelling artefacts, such as the icons that you want to represent your plugin and the other elements it can interact with.
The packages contain the deployment and runtime functional code of the plugin, such as product installation and lifecycle management scripts.
The transformers convert the application model (as drawn on the canvas) to the topology model (as deployed to the hardware) including the target location of packages.
There are three core sub-types that can be defined in a plugin:
Components contain product and application-specific configuration; examples are the EAR application component and the DB2 component.
Links represent a relationship between components (they are shown on the canvas as the blue lines literally joining components together).
Policies are the real “secret sauce” of application patterns; they can contain levels of complexity impenetrable to all but the most dedicated system administrators! At present, not many policies have been implemented; the most obvious (and arguably one of the most complex), is the scalability policy. This policy allows the user a choice of four different load metrics to control the dynamic scaling of WAS; it’s a really powerful policy and hopefully one that IBM will continue to invest in, to broaden its reach to other middleware products.
When you draw the application diagram on the canvas, a JSON document (the application model) is being created under the covers; it is visible in the Virtual Application Builder under the Source tab. Assuming you’ve created a coherent model, containing artefacts linked together correctly, you should be able to deploy it, which is the procedure I’m interested in for the purposes of this blog.
As previously stated, the PureApplication System doesn’t deploy the application model directly because there is no information relevant to physical manifestation. For example, what follows is a snippet of the Web Application sample that comes with a PureApplication System:
This snippet shows how the Web Application component on the canvas (the node type of type “WAR” in the model) has been configured: the user has asked for the webappSampleWeb.war file to be deployed with a specific web context root of webapp.
This is where you begin to realise that the PureApplication System is a framework for the plugins it contains: the transformation of the application model to the topology model. It is the topology model that dictates the creation and configuration of Virtual Machines. PureApplication System doesn’t understand how to create the topology model itself; this is delegated to each individual plugin that has been placed on the canvas. The plugins are invoked in order: components first (such as WAS and DB2), followed by the links between plugins and policies.
The diagram below shows the logical flow and also introduces a new element of the puzzle – Kernel Services. Kernel Services is the orchestration engine of PureApplication Systems. For the purposes of this blog I shall just show what it does in terms of the transformers and not discuss what else it does in PureApplication systems (there is, in fact, another element – The Storehouse – which we will ignore for now).
Each plugin must declare a transformer, whose role it is to convert the application model into the topology model for deployment (1 & 2). Each transformer is given access to its element of the application model, including any user-specified attributes (e.g. EAR file name) and, in the case of a component, the output from the transformer is likely to include Virtual Machine requirements – OS, memory, etc. – as well as software packages and application elements.
We’ve already seen how the application model represented the WAR component; what follows below is just a small part of the topology model document for an instance of the “Sample Web Application Only” application pattern, that IBM supplies with a PureApplication System:
(For any given deployment of an application instance, you can see the complete topology document – plus all the other JSON documents – in the storehouse browser. The Storehouse browser is located under the “System” tab in the PureApplication Systems web administration interface. The documents are held under User à Deployments à <deployment_id>.)
In the example above we can see that there is only one Virtual Machine being created (as there is only one vm-template element) and that there are a number of “parts” being deployed to that single VM (I’ve edited out many more parts that were irrelevant to this discussion!).
We can also observe that the “WAS” role has been put onto the VM; it was added by the WAR component transformer whose operation we have been examining. You will note that although the WAR file name was clearly visible in the original application model, it has now been substituted for the identifier “$$2”, which is a reference to another JSON file under the same deployed application tree: the parameter.json file. This contains the following entry, amongst many others:
So, that’s enough about component transformers: we’ve now got an idea of the level of complexity contained therein!
Link transformers are slightly different; they are driven by the outputs of the transformers of the components that they are connecting, i.e. the components’ topology models. They will then augment one or both of the component topology models, to create the deployable manifestation of the connection between the components (it is unlikely more VMs would be created, although this is certainly not prohibited). For example, in the case of the WAR-to-DB2 link, the link transformer manipulates the output of the WAR transformer – the topology model that the transformer created – and adds a WAS resource adapter installation and accompanying configuration packages into the WAS VM role.
Policy transformers are different again; policies may require their own independent VM in order to host a centralized policy management service. Alternatively, they may just need to configure some code on their dependent component.
In terms of implementation, the transformers are OSGI components which implement the com.ibm.maestro.model.transform.TopologyProvider interface, either in Java code or by providing a template: both implementations have access to the same components of the model, and are called at the same time in the deployment lifecycle. I won’t discuss those here – perhaps in another blog !
In Summary: there is a proliferation of new and re-used terms around PureApplication System, and some of them don’t relate obviously to the problem statement. I’ve discussed just a tiny bit of the system here: the relationship between what the user draws on the canvas (the application model) and how the deployed plugins turn this into deployable entities (the topology model).
John Hawkins is a Principal Consultant at Icon Solutions. He has seventeen years of IT industry experience and worked for IBM at their Hursley laboratory on MQ, Message Broker and cloud technologies . John is highly skilled in Messaging, Pure Application patterns and implementing Proof-of-concepts with bleeding edge technologies. He prototyped MQ and WESB on PureApplication Systems as well as inventing a new type of “business driven” scalability policy.
This post was written by John Hawkins, a Principal Consultant at Icon Solutions.
Although PureApplication System has some great local hot swap and off-site Disaster Recovery technology you still need to understand what it means for you and your application. I haven’t found any concise documentation for this so I’m sharing what I’ve found out here.
Pure has the ability to shut down the workloads on a compute node and move it to another Compute node. This is called “evacuation”. Evacuations can be planned or unplanned (you can guess the difference !).
“Why would I do a planned evacuation?” you may be asking yourself. Well, if you need to upgrade your compute nodes for instance. Or maybe it’s got an intermittent fault that needs looking at (it happens – even with IBM hardware).
Planned Evacuation
With PureApplication systems you can plan to close down a node. If a planned evacuation takes place then Pure will migrate your running vms to other nodes. In this case the Pure “magic” makes sure that your workload never knows what happened: it gets the same IP address, same disk etc. Now, that’s good, but the real sauce is that it mirrors the exact state of the VM when you took it down – that’s pretty cool ! To be clear: your complete memory, CPU, disk, network resources are migrated, even your tcp adapter/packet state is preserved with no data loss or discrepancy. This means that your application will be restarted on a new node and continue right where it left off – even if it was in the middle of a transaction !
One minor downside is that you may notice a slight decrease in performance for one or two minutes, but, if that’s the price you pay for not having to worry about a node being brought down then that’s cheap by my book.
Unplanned Evacuation
For an unplanned evacuation (failure) of a Node – Pure will still move the workload VMs to another node in the rack. However, in this case, the move won’t be so smooth. To the application, it will look like the VM has crashed and it’s been restarted (naturally enough because that’s what happened). There’s no attempt to reproduce active memory state or anything else. In this case, standard transaction semantics apply and, if you care that you don’t lose data, then you need to code around it. There are plenty of discussions on how to code transactions so I won’t go into that here.
Disaster Recovery
Not only can Pure do hot-swap and VM evacuation across a single Pure box but it can do Disaster-Recovery across two racks. Disaster-Recovery is usually between two distant data-centers which you don’t think will be affected by the same physical problems at the same time. Look at this video to see just how easy that can be setup: http://www.youtube.com/watch?v=YOJS6z18p7E . Disaster recovery is achieved using hardware based disk replication. When the primary rack fails all the workloads get moved to the new, standby, rack. Again, it looks to the application like the VM failed – but that’s acceptable – your rack’s just crashed! The application does have some of the same context though – it still has the same disks and IP addresses on the new, remote, rack. But you still need to code around any transactions you were in.
Middleware High-Availability
The functions just described are at the infrastructure level. Pure can also help you at the software level too. For instance, the in-built WebSphere Application Server application pattern will deploy two WAS nodes on two different compute nodes. This means that if the first compute node fails then the second one takes over while the first recovers (this may mean that the first instance gets moved to a different node remember – but all that’s taken care of for you). Be careful though, this does require you to set your scalability policy so that there a minimum of two WAS nodes in the cluster.
Summary
So, to summarise: Pure is IBM’s latest system of expertise. They’ve not only given it expertise in terms of the software patterns but they also help you manage the solutions at runtime too.
For a planned shutdown of a node you have no fears. Your solution is in very safe hands – the application will not know that it ever got moved nor will the clients that connected to it.
For a node failure the application will get rebooted, as if it had just started up, so you’re going to need to code around any data loss issues you may have. This applies to whether the failure was a local inter-rack failure or a planned or unplanned Disaster-Recovery (cross-rack) scenario.
John Hawkins is a Principal Consultant at Icon Solutions. He has seventeen years of IT industry experience and worked for IBM at their Hursley laboratory on MQ, Message Broker and cloud technologies . John is highly skilled in Messaging, Pure Application patterns and implementing Proof-of-concepts with bleeding edge technologies. He prototyped MQ and WESB on PureApplication Systems as well as inventing a new type of “business driven” scalability policy.
John Hawkins is a Principal Consultant at Icon Solutions. He has seventeen years of IT industry experience and worked for IBM at their Hursley laboratory on MQ, Message Broker and cloud technologies . John is highly skilled in Messaging, Pure Application patterns and implementing Proof-of-concepts with bleeding edge technologies. He prototyped MQ and WESB on PureApplication Systems as well as inventing a new type of “business driven” scalability policy.
In these two videos, Tony Barker provides an overview of virtual system patterns and virtual application patterns. He shows examples for each type of pattern and provides perspective on how you can decide which type of pattern best fits your needs.
This Thursday, September 12 at 11 AM EDT, the GWC will host a webcast: Meet the new “mini” Power-based PureApplication System and latest IBM Patterns.
Planned pattern topics for this webcast are new IBM patterns covering the newer version of our BPM solution, as well as Sharepoint and MySQL solutions. This session is free to attend. You can register on the GWC website.
If you’re not able to attend the session live, you can listen to the recording. To access the session recordings, go to the Recent Archives tab on the Webcast page.
Many of the posts so far have covered pattern types that were created by IBM or by Business Partners. You can also create your own pattern types!
Dominique Vernier wrote an excellent post on creating pattern types for the Expert Integrated Systems blog. You can read it here: Your own pattern-type in few steps!
You can find all of the related posts by looking for “Pattern-type (Part x)” in the title of subsequent posts. I found it easiest to read the whole series in order by going to the Index tab and accessing each post in the series from there.
Did you know that the IBM PureApplication System Information Center provides topics about virtual application patterns such as the Web Application Pattern? With this pattern, you can build and deploy web applications with components such as Java EE applications, databases, LDAP servers, messaging, and more. This pattern is included with your PureApplication System license. See the information center for details.
And while you’re in the information center, why not provide some feedback about the content? You might have suggestions for improvement, or perhaps you’ve identified gaps or places where you’d like to see more details. Just click the Feedback link at the bottom of each topic to send the writing team an email. We really want to hear from you!
This post was written by Ewan Withers, a Senior Consultant at Icon Solutions.
To demonstrate the capabilities of the PureApplication System, we built a test-bed application: an Internet Banking demo, based on a client’s specification (a standard multi-tier architecture, with servers running HTTP Server, WebSphere Application Server, WebSphere Message Broker and DB2). We then created an instance of the two types of Pattern:
Virtual System Patterns (VSPs): script-configured, VM-centric, static deployments with a traditional administration model.
Virtual Application Patterns (VAPs): application-centric “expert” components, with policy-driven automatic scaling and a Cloud-based administration model.
Porting a complex, multi-component application into a VSP should be achievable within current development frameworks; you may even be able to leverage existing deployment artifacts. Any changes required are of the same order as you would expect when performing any other kind of migration (e.g. moving from Windows to Linux, or upgrading WebSphere Application Server). We found that the benefits of the VSP can even be realized during its own development: rapid, on-demand identical deployments of Development systems accelerated the VSP’s own creation.
A radical paradigm shift with long-term benefits
Wrapping the same application in a VAP, in contrast, requires a radical paradigm shift: you need to think in Cloud-centric terms because of the opacity of the deployed VAP. A considerable amount of development effort is required, and during development of the VAP we found that there is a “sweet spot” somewhere between the following endpoints (which will vary depending on the application in question):
Complete re-build of the application, and use of out-of-the-box VAP plugins
Implementation of complex customized VAP plugins, and no changes to the application
The strategic application architecture plan can include movement along this continuum, with the investment being justified by the long-term benefits of the Cloud-centric deployment. As well as realising the full benefits of the PureApplication System, the VAP approach can enforce greater rigour in development standards, bringing about a general improvement in code quality.
A PureApplication System is a major long-term investment, and maximizing that investment is largely dependent on choosing the right corresponding application development strategy. The choices of VSP or VAP (or a phased approach between the two) are all entirely valid; indeed, the choice might vary for different applications – and whatever the decision, the benefits will be manifold.
We were delighted when our pattern won an Impact 2013 PureApplication Award – an award which recognises the best innovative pattern. If you’d like to find out more, or see a demo on our pattern, contact me at ewan.withers@iconsolutions.com.
Ewan Withers is a Senior Consultant at Icon Solutions. He has sixteen years of IT industry experience and has worked with the IBM WebSphere software portfolio in both software development and solution implementation. Ewan is highly skilled in enterprise and integration architecture, solution design, product development and performance engineering.
If you want to see a virtual system pattern in action, take a look at this video: http://www.youtube.com/watch?v=IPsvRFua0z4. It’s only 6 minutes long, and it shows a pattern builder as he designs and deploys a virtual system pattern (VSP) on PureApplication System for an application from Saba Software.
WebSphere Operational Decision Management (ODM) provides decision making and change detection tools for business. You can purchase the IBM Operational Decision Manager Pattern to run ODM on your PureApplication System. This pattern is a virtual system pattern and is an example of the virtual system patterns that Andre described in the previous post.
I think the article provides a great overview of the pattern, instructions for installing and deploying it on PureApplication System, and how to work with the instance after it is deployed.
If you want to purchase the pattern, you can find it here on PureSystems Centre: IBM Operational Decision Manager Pattern. While you’re there, you can watch a YouTube video about the product that is embedded in the page.
IBM Cloud Pak System 