Chapter 3: Overview of
BizTalk Server
by Rand Morimoto
In This Chapter
Step 1: Determining the Interchange Goals and Objectives Step 2: Modeling the Business Process Step 3: Agreeing on Process Specifications and Parameters Step 4: Creating the Mapping Between Input and Output Data
Formats Step 5: Configuring the Triggering Mechanisms to Process
Information Step 6: Completing the Schedule for the Transaction Process
Instead of listing the features and functions of the various technologies and
tools built-in to the BizTalk Server product in a glossary/definition format,
this chapter is organized chronologically by the process you follow to implement
a BizTalk Server solution. Describing the BizTalk Server process flow involves
defining goals and objectives of data interchange, modeling the business
process, creating the specifications, and creating the maps, before finally
confirming the trigger mechanisms and schedule for transaction processing.
In this chapter, BizTalk Orchestration is not purely defined, but instead
illustrated how it relates to the business process model. And BizTalk Mapper is
described in this chapter in terms of how it translates business processes into
data interchange maps.
A BizTalk Server process flow starts with
Understanding the goals and objectives of the organizations that want to
exchange information Modeling the business process from start of the flow to
completion Agreeing on the specifications and parameters involved in the
process Creating the mapping between input and output data formats Confirming the triggering mechanism to process the
information Completing the schedule for the transaction
processes
When the process flow is followed, BizTalk Server can solve business
information exchange challenges. The full process requires modeling business
procedures, analyzing data communication formats, mapping formats, and properly
configuring BizTalk Server. This chapter provides an overview of the sequential
steps involved in the process and refers to the chapters throughout the balance
of this book that highlight the steps in more detail.
Step 1: Determining the Interchange Goals and
Objectives
Although BizTalk Server is an excellent solution for business to business
(B2B) and Enterprise Application Integration (EAI), a successful implementation
of BizTalk Server requires business analysis and goal setting to ensure that the
solution meets the needs and objectives of the stakeholders of the project. This
first step involves asking the right questions and gathering the information
necessary to document the overall expectations for the project. A small amount
of planning will go a long way in coming up with a complete end solution.
Who Understands the Interchange Goals and
Objectives
When determining the interchange goals and objectives, it's important to
understand who the project's stakeholders are who ultimately make the
determination of whether the project is successful. Unlike many internetworking
projects where the decision maker is frequently someone in the Information
Technology (IT) department, for a BizTalk Server integrated solution, the
decision maker is frequently the CFO, COO, or a Line of Business (LOB) manager.
It is important to know who owns the results of the project. This individual or
group of individuals will understand the overall goals and objectives for the
data interchange needs of the project.
The stakeholders for the project also understand critical business factors
necessary to know before and during the project. It is common that the
stakeholder is also the person who would approve the budget and get personnel
resources allocated to support the implementation and ongoing maintenance of the
project.
Some stakeholders are hands-on and involved right from the beginning of the
project and throughout the entire length of the project. These stakeholders will
be directly informed on the project, process, and tasks to make decisions on
budget and project engagement firsthand. Other stakeholders assign the
information gathering and analysis tasks to someone else and only request that a
final report be prepared and submitted for review. The stakeholder will
typically still be the individual responsible for making the final decision, but
others will be in a position of influencing the final decision. Their report and
recommendation to the stakeholder would be biased by their opinion of the
solution. In the cases where there are both stakeholder(s) and influencer(s),
multiple individuals would need to be involved in the goal and objective
definition process.
What Are the Interchange Goals and Objectives
With the stakeholder(s) and decision maker(s) identified, the goals and
objectives for the project need to be understood. Determining the goals and
objectives might be as simple as asking a single individual, or it might require
a group meeting to come to a consensus. The data interchange goals and
objectives vary from project to project. There are typically business goals and
objectives, technical goals and objectives, and operational goals and
objectives.
After the goals and objectives are understood, they should be written down to
create a documented functional specification. This functional specification
should then be approved and "signed off" by the stakeholders of the
project to get a confirmation of their understanding of the goals and objectives
of the project.
Business Goals and Objectives
The business goals and objectives for a data interchange project are those
that impact or influence business processes. Examples of business goals and
objectives would include
Publish a part number and description catalog every week with updated
infor-mation. Automate order processing and acknowledgement confirmation
procedures. Decrease overall costs by automating order and transaction
processing. Improve response time by eliminating manual processes with automated
processes. Enhance transaction scalability by streamlining automated
processes.
Technical Goals and Objectives
The technical goals and objectives for a data interchange project are those
that impact or influence IT operational processes. An example of technical goals
and objectives would include
The consolidation of EDI and HTTP transaction systems into a single
server system Improving logging and tracking of transactional information so that
weekly reports can be created Standardizing the development language to be XML-based
Operational Goals and Objectives
The operational goals and objectives for a data interchange project are those
that impact or influence timelines, budget, or operational processes. An example
of operational goals and objectives would include
The completion of the implementation of the project before the start of
the next fiscal year The completion of the planning, programming, prototyping, testing, and
imple-mentation of the project for under $100,000 The redeployment of idle personnel from a completed project to a new
project
What Information Needs to Be Exchanged
With the stakeholders identified and the business, technical, and operational
goals documented, the organization needs to define specifically what needs to be
exchanged. This includes identifying the input format of the information coming
into the BizTalk Server (such as EDI, HTTP, HTTPS) and the output format of the
data being exchanged.
Also during the information gathering step, determining what applications
will be involved and the transaction formats supported by the application need
to be identified. There's a difference in how the BizTalk Server is
configured and managed when two inventory control systems are connected versus
the connectivity between a customer relationship management (CRM) application
and an inventory control system. Each system has a defined format and method of
communications.
There are several expectations for any implementation, whether it is an
expectation of time, budget, scope of work, ongoing maintenance, or support
arrangements to name a few. These expectations tend to drive behavior.
What Is the Need for Scalability and Integration
Finally, when determining the interchange goals and objectives, understanding
the need for scalability of the BizTalk Server solution helps the organization
plan the infrastructure needed to support the implementation. This involves
planning and budgeting for multiple servers to support the demand for system
transactions, or it might involve having multiple servers provide for a
failover/fault tolerance system. An analysis of the needs and expectations of
the stakeholders should assist in determining the type of programming needed to
facilitate redundancy and performance metrics.
Additionally, an organization would want to know whether the BizTalk Server
solution needs to integrate with an existing Windows environment or whether it
would be tightly integrating with a non-Windows environment. An integration
demand analysis would help the organization determine whether the tools
available for development within a Windows-based environment are applicable and
need to be acquired, or the support for multiple users can be identified and
implemented in a native format or have non-Windows integration requirements.
Step 2: Modeling the Business Process
From the first step, we were able to interview the stakeholder(s) and/or the
decision maker(s) to understand the business, technical, and operational goals
and objectives of the organization. This information now needs to be modeled
using the tools included with BizTalk Server to begin translating the process
into code that can be run on the BizTalk Server.
This step involves identifying what needs to be processed, predicting what
responses need to be returned, determining what events are sequential and what
events trigger other events, and modeling the end-to-end information flow. This
information is then entered in to a BizTalk Server tool called the BizTalk
Orchestration Designer.
How to Describe the Business Processes
Key to the success of this step is translating the goals and objectives of
the business into steps that can be followed methodically by the BizTalk Server.
Effectively, a decision tree needs to be created where every yes/no or
success/failure is identified, and appropriate loops are documented. If a group
of planners is involved, this process may be done on a whiteboard.
If the organization already has a manual process that is desired to be
automated, one big decision the organization needs to make is whether to mirror
the existing process or rethink the entire process. Although an existing manual
process might work fine, it might be laden with several steps that have no tie
to any tangible function anymore. This might be a sign-off or verification
process needed in a manual system but that can be bypassed in a fully automated
system. It might be a process that involves checking or double-checking data
entry information that might not be necessary in a single entry automated
process. The business process steps need to be reviewed to determine whether
they are necessary in an automated system.
Overview of BizTalk Orchestration
Unlike many IT-focused systems, BizTalk Server does not jump the organization
straight from high-level business needs right into programming code. BizTalk
Server includes a step called BizTalk Orchestration that allows an organization
to define its business processes in a graphical format that can then be modeled
and linked to more complex data processing models. This allows a separation
between the definitions of a process and the implementation of the process. This
enables different parts of the business process to be modeled by those who know
the processes the best (the business-focused individuals). Those processes then
are compared with the technical programmability parts of the process.
BizTalk Orchestration Designer
The tool that facilitates the modeling process is called the BizTalk
Orchestration Designer. The tool is effectively a Visio modeling tool that
provides a way for an organization to model its processes into a flowchart type
drawing. This drawing is called an XLANG scheduling drawing. Most business
managers understand the concept of BizTalk Orchestration Designer because they
are familiar with flowcharts and flowchart processing, so having this
orchestration design step helps to ensure that the technical process mirrors the
expected business goals and objectives.
Note - Although a business manager
usually understands flowcharting and will be able to review the models created
in the BizTalk Orchestration Designer, typically a technical expert would be
involved in the actual creation of the BizTalk Orchestration Designer model.
BizTalk Orchestration and the BizTalk Orchestration Designer are covered in
more detail in Chapter 9, "Introducing BizTalk Orchestration."
Some of the BizTalk Orchestration Designer process options include Action,
Decision, While, Abort, Fork, and Join. These options enable different flow
routines to occur.
Action
An Action defines a process that typically requires information to be
manipulated. It might be an action step that runs the information through a
filter, or it might be an action step that reformats data being processed.
Decision
A Decision defines a process where a yes/no, data query, or other information
analysis is conducted, and the result determines which branch to follow to
continue with the workflow. It may be determined that if a field is blank to
have a branch that enables an action to fill the field with data. However, if
the field is not blank, then a branch would allow the process to continue.
While
The While object allows a process to loop until a specific event occurs. This
might be a process that continues until the end of the dataset is reached, or it
might be a process that continues until a specific data record is reached. A
branch can then be followed to a subsequent decision or action.
Abort
An Abort object identifies a step where the process needs to be terminated.
In many cases, the abort becomes a fail-safe test that determines that the
information received is malformed, signaling data corruption in the transmission
or just that incorrect data was transmitted. This step can provide a way to stop
a process from continuing with incorrect information.
Fork and Join
A Fork splits a process to conduct simultaneous tasks that can then be Joined
later in the process. This is frequently a process that enables information to
be gathered for decisions to be made. For example, a fork in a process can be
created to go out and gather price and availability information from multiple
vendors. This information can be gathered and processed simultaneously. From
this fork, when the information has been analyzed, the system can join back
together to continue with the process.
Step 3: Agreeing on Process Specifications and
Parameters
After the models have been created and the processes defined and agreed on,
the next step is to agree on the dspecifications and parameters for the
information interchange. This step gets more involved in the technical data
formats of the information.
Specifications are created and can be based on industry standards, such as
XML, X12 EDI, SAP IDOC, or UN/EDIFACT, or the information can be structured
simply as flat files, such as delimited, positional, or delimited and positional
formats. From this data format and specifications the appropriate filters can be
applied to the fields of input and output information.
The parameters of the data structure typically define what information to
expect within the specification format. This could be as simple as knowing that
the first fields in the dataset include name, address, and purchase order
number, or the parameters might specify that the first fields in the dataset are
raw order data such as quantity and agreed-on pricing. It is important to know
the data format and the structure of the data so that the right information is
automatically extracted and processed properly.
BizTalk Editor
The BizTalk Editor is a graphical tool that comes with BizTalk Server that
helps with the creation, editing, and management of document specifications. It
allows for the creation of these document specifications by defining the fields
and records based on the needs of the organization using BizTalk Server. The
specifications can be created by manually defining the records and fields in the
BizTalk Editor tool, or by using any of the built-in specifications for the
support of standards such as X12 EDI, UN/EDIFACT, XML Data-Reduced (XDR), and
the like.
By defining or modeling the format of the document in the BizTalk Editor
tool, the BizTalk Server can parse and serialize the output data regardless of
the type or format of the information. Additionally, after the format has been
defined, an instance of the data can be tested to verify that the specification
is correct.
Although transparent to the user, BizTalk Server represents documents in an
XML Schema Definition Language (XSDL) format. The document specifications define
the attributes for the data, whether data fields are required or optional,
whether the data values are fixed length or variable, and the like.
Simply, the BizTalk Editor is used to create and validate specifications to
be used for reference purposes. After the specifications have been defined, data
can be tested on the server to ensure that the specification is correct.
Step 4: Creating the Mapping Between Input and
Output Data Formats
Although the BizTalk Editor can structure the information into common formats
such as X12 EDI or UN/EDIFACT that can be read and processed by the system that
understands the standard formats, many times information is received and needs
to be translated into a different format. This mapping of input and output
fields ensures that the right information is relayed to the destination.
The BizTalk Mapper is the tool included with BizTalk Server that provides the
mapping between fields.
BizTalk Mapper
BizTalk Mapper allows the definition and the correlation of data between
records and fields for one document specification, and the records and fields
for another. The mapping of information from one input format to another goes
through one of two different processes. The BizTalk Server can alter the schema
of the information in a process that is said to transform the information, or
the BizTalk Server can actually alter the data itself in a process that is said
to be a translation of the information.
The transformation process used by the BizTalk Server is handled by script
coding using functoids. A functoid is a reusable function built-in to the
server engine that enables simple as well as complex structural manipulation
between source and destination specifications. A transformation of information
is common because input and output formats frequently do not match and need to
be mapped so that fields from one format match the fields of the other format
during the transformation process. A transformation process, although by
definition means that the data is altered, does not necessarily mean that the
information is deleted and replaced with completely different data, but that
information is analyzed and modified for a new or different format.
An example of data transformation could be the replacement of a State
definition such as Calif with a standard two-letter CA postal format, or a data
label may be padded with extra spaces to keep the data format the same length
such as keeping a field to eight total characters even if that means that
several characters are spaces. BizTalk Mapper uses extensible style sheet
language (XSL) to process the map transformation.
As an XSL generator, the BizTalk Mapper allows for both simple one-to-one
mappings and more complex mappings using functoids. The graphical mappings
defined in the editor are compiled down to XSL to be applied by the BizTalk
Server engine at runtime to the inbound document. Inbound documents are parsed
into XML representation of the information using the specifications defined in
the BizTalk Editor. If mapping is required, the XSL is applied, and the document
is serialized into the correct outbound format by the messaging engine using the
outbound BizTalk Editor-generated specifi-cation.
Step 5: Configuring the Triggering Mechanisms to
Process Information
After information is formatted with a common interchange format, the data
needs to be transported between organizations or systems. The triggering
mechanism that initiates the process to transport data is called BizTalk
Messaging. BizTalk Messaging provides the capability to send data in a secure
and reliable manner between systems. Because BizTalk is used for both external
and internal data transport processes, the transport of information may be
between external trading partners through a VPN tunnel over the Internet, or the
transport of information may be between internal accounting and client
management systems on a corporate LAN.
Overview of BizTalk Messaging
BizTalk Messaging takes information and transports it from a sender to a
receiver. BizTalk Messaging is similar to a traditional e-mail system; however,
an e-mail system assumes that a message comes in using a standard message format
(such as SMTP) and transports it to a recipient using a standard (SMTP) message
format. As highlighted in Chapter 1, "Motivation and Uses for BizTalk
Server," BizTalk Messaging accepts data in a variety of incoming formats
(such as EDI, HTTP, HTTPS, and so on), takes the programmed field map created
in BizTalk Editor, and maps the data fields to an output format using BizTalk
Mapper to create the transported BizTalk message.
BizTalk Messaging is covered in more detail in Chapter 6, "Introduction
to BizTalk Messaging."
Data Stores
In a BizTalk Server environment, information is stored in data stores that
are nothing more than SQL databases. The data stores hold the transaction data
and more importantly the transaction logs. At any time, through the use of the
document tracking utility built-in to BizTalk Server, reports can be generated
using the data and log information to validate that information has been sent
and received. The information can be verified to ensure accuracy, time stamps,
and other critical confirmation of data transaction.
The Shared Queue (SQ) Database
One of the data stores in a BizTalk Server environment is the Shared Queue
(SQ) database. The Shared Queue database is a SQL database shared by all servers
in a BizTalk Server group. If there is only a single server in an organization,
the Shared Queue tracks only one server. If there are multiple servers for a
BizTalk Server group, all the queue information is stored and tracked on a
single server.
The Shared Queue database also tracks checkpoints for data transactions. The
checkpoint is a file that notes the transaction status of a BizTalk Server
system. If one server in a group fails, other servers can take over the process
tasks, such as receiving messages, posting messages to the work queue, and
transporting messages to destination servers. In a way, the failover process
facilitated by the checkpoints in the Shared Queue database creates a load
balancing and built-in fault tolerance system for the BizTalk Server
environment.
The BizTalk Messaging Management Database
Another SQL database critical to the operation of a BizTalk Server
environment is the BizTalk Messaging Management database. The BizTalk Messaging
Management database stores information related to server configurations, such as
group and server settings and messaging configuration settings. The information
in the database either is entered using the BizTalk Messaging Manager interface
or is programmatically entered using an API. This database effectively stores
the system configuration information for the server.
Send and Receive Architecture
As clarified throughout this chapter, BizTalk Server was built with
variability so that there is flexibility on the process and data format to send
and receive information from a variety of sources and destinations. The
architecture enables disparate systems to communicate with each other, and the
simplicity of message translation and transformation provides the capability to
interconnect multiple systems with the same set of data.
Pass-Through Submission
One method of data transport is called pass-through submission.
Pass-through submission allows data to bypass the parsing, decoding, decryption,
transformation, and signature verification stages of a BizTalk Server process.
Data is passed directly to the destination noted in the submission parameters.
Pass-through submission is commonly used to transmit binary files, such as Word
documents, applets, structured database files, and so on that do not require
translation.
BizTalk Server Administration
For managing and administering the BizTalk Server environment, a BizTalk
Server Administration tool is included. The BizTalk Administration tool is a
Microsoft Management Console (MMC) snap-in that shows the Shared Queue databases
on a BizTalk Server as a series of queues. Also within the administration tool,
an administration can make modifications to the configuration of the server such
as setting tracking information, group interaction, messaging transport, queue
management, or database replication.
BizTalk Server Administration is covered in Part VI, "BizTalk Server
Administration."
Step 6: Completing the Schedule for the Transaction
Process
The first five steps of this process involved identifying the business,
technical, and operational processes and then using BizTalk Orchestration to
graphically represent the processes. The process continued with the creation of
the specifications for the data and then mapping the information between trading
partners and systems. This last step involves completing the XLANG schedule
drawing by compiling it into an XLANG schedule that can be tested and then put
into production.
Note - Not every application use of
BizTalk Server requires the implementation of both Messaging and Orchestration
services. Depending on the input and output format demands of the transaction,
due to the overlap in functionality of these two services, an organization may
find that only the Messaging or Orchestration service is required.
The BizTalk Orchestration XLANG Scheduler
Engine
BizTalk Server comes with a service called the BizTalk Orchestration XLANG
Scheduler Engine. This service controls the activation, execution, dehydration,
and rehydration of an XLANG schedule. This process takes the final agreed-on
business and technical process and puts it into a format that can be executed on
the BizTalk Server.
The XLANG schedule defines the steps to be performed, the components that
must be called, and the data that will pass through the BizTalk Server to
fulfill the defined transaction process. When documents are sent from the XLANG
schedule to the BizTalk Messaging Service, the name of a channel that will
receive the documents must be defined. After the channel is defined, documents
can pass from the BizTalk Messaging Service to the BizTalk Orchestration
Service.
When in the BizTalk Orchestration Service, the document is either processed
in XML format, or if the document is not in XML, the XLANG Scheduler Engine will
need to embed the document in the engine's standard XML wrapper. When
inside the XLANG schedule, the document can undergo any defined modifications.
After processing the document, it can then be sent back to BizTalk Messaging or
it can be sent out of the XLANG schedule to a private or public queue.
Schedule activation and execution are the processes that enable the schedule
to be run and then ultimately be executed to process information. When schedules
are unused, they are dehydrated and then dynamically rehydrated when needed.
This dehydration/ rehydration process minimizes the need to allocate resources
of the BizTalk Server to processes that are unused, allowing for the
optimization of the system to handle the needs of the transaction
environment.
Summary
BizTalk Server is a powerful business transaction product that, when properly
planned and implemented, is of help to organizations conducting B2B and EAI
tasks. Several technologies are embedded, and several tools are included with
the BizTalk Server product, including the BizTalk Orchestration Designer,
Editor, Mapper, Messaging Manager, and Server Administration MMC utilities. This
chapter introduced the tools and technologies and how they fit into a BizTalk
process flow and provided references to more specific chapters throughout the
book.
© Copyright Pearson Education. All rights reserved.
|