As a BizTalk Server administrator you will need to have tools for monitoring the BizTalk environment, deployment of applications, and troubleshooting. Out of the box, BizTalk comes with the Administration Console. This is a powerful tool and it lets you do a lot of things when it comes to operating and troubleshooting. In my view the BizTalk Administration Console is more focused towards administrative tasks than monitoring your BizTalk group.
The Administration Console alone does not entirely do the job for you in regard to BizTalk operations. There are many manual tasks that need to be performed. One example is importing an application and configuring it. This is on the deployment side. Neither does it help in certain scenarios when there is an issue with sending a message to another system. When it comes to troubleshooting, there are times when it will not help in resolving an issue or finding the root cause. And when it comes to monitoring, deployment and troubleshooting, there are many tools and products available that can make life easier for a BizTalk administrator.
The BizTalk administration console is not suitable for monitoring BizTalk, generating alerts and notifications. Commercially there are products and tools available that can fulfill motoring requirement for an enterprise/organization in a much better way. These product/tools are:
- System Center Operations Manager
- HP Open View
- Integration Platform Monitor
- Frends Helium
- Codit Integration Dashboard
- AIMS Innovation
Microsoft offers System Center Operation Manager (SCOM), which enables overall end-to-end monitoring of your entire enterprise IT infrastructure and its applications (i.e. other server products like BizTalk, Exchange, and third party software). Through a single UI an administrator and/or engineer can monitor the health, performance and availability of data center environments – across applications, operating systems, hypervisors and even hardware through usage of management packs. SCOM uses the term management pack to refer to a set of filtering rules specific to some monitored application like BizTalk Server. This management pack after importing into SCOM offers two views one intended for BizTalk administrator/operator and one for Enterprise IT administrator:
- Application View: A BizTalk administrator or operator is able with this view to monitor the state and health of various BizTalk Server artifacts and applications such as orchestrations send ports, receive locations, and so on.
- Deployment View: An enterprise IT administrator is able with this view to monitor the state and health of the various enterprise deployments the machines hosting the SQL Server databases, machines hosting the SSO service, host instance machines, IIS, network services, and so on
HP Open View provides software for large-scale system and network management of an organization's IT infrastructure. It is a suite of business computer management programs from Hewlett-Packard (HP). IT professionals can use Open View to manage applications, device availability, network conditions and status, system performance, service and program maintenance, and storage resources. This is quite similar to what SCOM has to offer.
Monitoring BizTalk can be done using SCOM or HP Open View by administrators and issues can be diagnosed and resolved using BizTalk administration console with or without combination of other tools. To use SCOM solely for BizTalk monitoring might be too costly for mid-sized organization or even an enterprise. The product requires highly skilled people to implement and operate SCOM, and it has a steep learning curve. Besides the skillset it also requires an investment in a fair amount of hard- and software. Since SCOM can monitor any Microsoft product, Azure and third-party products it would be waste of resources to just use SCOM for monitoring BizTalk. SCOM be a perfect combination with some tools listed in this post and/or HP Open View.
I am not going to discuss each of the alternatives, however I will mention that one of the most innovative and feature rich product is BizTalk360. BizTalk360 is a web based BizTalk monitoring and support tool for Microsoft BizTalk Server. It addresses some of the common problems customers face today like governance/auditing, fine grained authentication, remote access and so on. The product has been developed by Saravana Kumar and is currently available in version 4.0. BizTalk360 supports version 2006 and up. It can be downloaded from the product site and is easy to evaluate on any BizTalk development environment.
One of the important tasks for a BizTalk administrators concerns deployment:
- Importing, configuring and starting applications.
- Maintaining and building bindings, scripts and packages.
- Un- and redeployment of applications.
- Maintaining and configuring adapters.
All of these tasks can be down with the out-of-box capabilities of BizTalk. When a developer builds a BizTalk solution and deploys it to his own BizTalk environment he can export it from the BizTalk Administration Console. An MSI and binding file (XML) are extracted and these can be viewed as a complete “backup” of the application. These msi’s and bindings are handed over the administrator. He/she does not have to rely on Visual Studio to deploy straight from the code as the ability to create an MSI is offered through BizTalk Management Console.
The Back drawback with MSI is that it represents a certain version, and when outdated an extraction of right version is required. Deployments to different stages (test, acceptance, and production) are manual tasks and require some discipline by administrators. Besides that, versioning is important to keep a track of different versions of the application. Discipline by administrators is key when building and maintaining MSI’s. Therefore they may revert to using their own scripts, BTDF or third party tool.
An administrator can also use BTSTask.exe found at <install path>:\Program Files (x86)\Microsoft BizTalk Server 2010. BTSTask.exe is a command line utility that can be used to for performing many application deployment tasks from the command line like such as:
• Exporting an application and its artifacts to an .msi file by using the ExportApp command.
• Exporting binding information to an .xml file by using the ExportBindings command.
• Importing an application from an .msi file by using the ImportApp command.
• Importing binding information from an .xml file by using the ImportBindings command.
One of the power tools when it comes to deployment is PowerShell. It is a Microsoft automation framework, consisting of a command-line shell and associated scripting language built on top of, and integrated with the .NET Framework. PowerShell provides full access to COM and WMI, enabling administrators to perform administrative tasks on both local and remote Windows systems. With PowerShell you can automate the BTS tasks.
PowerShell can aid you in performing the deployment tasks in a more proficient, efficient, less error prone and time saving manner. Through this blog you can find some interesting posts on this subject, including ways to customize your deployment using PowerShell:
- Custom BizTalk Deployment - Part 1
- Custom BizTalk Deployment - Part 2
- Custom BizTalk Deployment - Part 3
There is however another alternative to perform deployments by using the BizTalk Deployment Framework (BTDF) for instance. Randy Aldrich has written some guidance on that part. The BTDF is a community tool provide through Codeplex. There is also a commercial tool available that can aid in deployment of applications called Nolio Zero Touch Deployment.
Out of box capabilities are good enough when you have a small BizTalk Group and a few not too complex applications. As soon as your BizTalk Group is growing larger or is more complex and the number of applications is growing (more than 5) than you may have to look for other options.
First source you can look for when there are issues with BizTalk is the Windows event log. The event log can be viewed from within the BizTalk Administration Console using the event viewer. This can give you some information of the issue at hand. BizTalk Server writes errors to application event log each time an error occurs. Error information in event log contains information on when, why, and where the fault happened. Through the BizTalk Administration console event log can be viewed and the administrator can start investigating the problem all within the same snap-in using Console’s Group Hub Page.
Yet the administration console may not be sufficient to find the root cause of your problem. For instance when messages are being sent or received by BizTalk. Messages cannot be received by another system, SSL fails, message is not reaching BizTalk through HTTP receive, and so on. Besides communication there can be issues with performance. Luckily there are other tools that can aid you in troubleshooting like:
- Network Monitor
- Processor Explorer and Process Monitor
- BizTalk Best Practice Analyzer
SoapUI is the leading desktop application for inspecting, invoking, monitoring, simulating/mocking as well as functional testing, security testing, load testing, compliance testing and surveillance testing of REST/WADL and SOAP/WSDL-based Web Service. The following resources can be used to download the tool, find guidance in usage and background information:
Fiddler is a Web Debugging Proxy which logs all HTTP(S) traffic between your computer and the Internet. Fiddler allows you to inspect traffic, set breakpoints, and "fiddle" with incoming or outgoing data. Fiddler includes a powerful event-based scripting subsystem, and can be extended using any .NET language. The following resources can be used to download the tool, find guidance in usage and background information:
WireShark is a network protocol analyzer for Unix and Windows. The following resources can be used to download the tool, find guidance in usage and background information:
Network Monitor 3.4 is a protocol analyzer. It allows you to capture network traffic, view and analyze it. The following resources can be used to download the tool, find guidance in usage and background information:
Windows Performance Monitor (Perfmon)
You can use Windows Performance Monitor to examine how programs you run affect your computer's performance, both in real time and by collecting log data for later analysis. See BizTalk Performance Counters and Identify Performance Bottlenecks.
The PAL (Performance Analysis of Logs) tool is a tool that reads in a performance monitor counter log and analyzes it using known thresholds. PAL tool reads a performance monitor counter log (any known format) and analyzes it. Basically it automates the analysis of performance counter logs, for instance one that is generated by the Bench mark wizard or PerfMon. In end you can generate an HTML or XML based report that graphically charts important performance counters and throws alerts (in red), when thresholds are exceeded. The thresholds are originally based on thresholds defined by the Microsoft product teams, including BizTalk Server, and members of Microsoft support
The unique capabilities of Process Explorer make it useful for tracking down DLL-version problems or handle leaks, and provide insight into the way Windows and applications work. Process Monitor is an advanced monitoring tool for Windows that shows real-time file system, Registry and process/thread activity. It combines the features of two legacy Sysinternals utilities, Filemon and Regmon, and adds an extensive list of enhancements including rich and non-destructive filtering, comprehensive event properties such session IDs and user names, reliable process information, full thread stacks with integrated symbol support for each operation, simultaneous logging to a file, and much more. The following resources can be used to download the tool, find guidance in usage and background information.
The BizTalk Message Box Viewer also known as Message Box Viewer (MBV) was created by Jean-Pierre Auconie. This tool retrieves information from a BizTalk System and identifies all possible issues, which could be critical or need attention, and present them in a user friendly format. This tool is extremely useful not only for troubleshooting, but also to do regular health checks.
The BizTalk Server Best Practices Analyzer (current version V1.2) performs configuration-level verification by reading and reporting only. The Best Practices Analyzer gathers data from different information sources, such as Windows Management Instrumentation (WMI) classes, SQL Server databases, and registry entries. The Best Practices Analyzer uses the data to evaluate the deployment configuration. The Best Practices Analyzer does not modify any system settings, and is not a self-tuning tool. Like the MBV this tool can be useful not only for troubleshooting, but also to do regular health checks.
This post shows that the toolbox of a BizTalk Server Administrator is rather versatile. The BizTalk administration console is not the only tool he/she has to work with. To be more productive the tools mentioned can be very valuable and aid in achieving that goal. Working with some of the tools is pretty straightforward, yet some require you to have more knowledge and time. Time you have to invest in learning the capabilities of the tools. However, this will be worthwhile!