Marcus Rosen.NET

As I recently upgraded my phone plan and got a Nokia N95 8G (which by the way is a fantastic device), I trailed the net for exciting and useful/useless applications to install.

One quite useful application that I can a cross is the Nokia Step Counter by Nokia Labs. It uses the accelerometer built into the phone and transforms the phone to a pedometer (step counter). The pedometer is quite accurate and I now now how much (or little) I move around in a given day.

Working in the IT industry and almost turning 30 I guess it's important to start thinking a bit more about my overall well being (read weight) and this actually seems to motivate me to walk a bit more.. ;-)

The application has a couple of cool features like auto start in the morning, diary, trends charts, etc.

You can download it from: http://www.nokia.com/A41033364

The feature has probably been around for years but I just discovered it the other day and already find indispensable.

If you are traversing through code and use the "Go to Definition", Visual Studio offers Back and Forward History similar to a Web browser so you can jump back and forth between code files and positions without scrolling. This is a very useful feature especially if you work with code that is unfamiliar and/or many levels deep.

Keyboard Shortcut:

Back: Ctrl + Minus
Forward Ctrl + Shirt + Minus

EDIT: Post has been update after comments by Rick Negrin (Microsoft).

The other day I was asked about the differences about SSIS and BizTalk as there are some blurred boundaries between these products. Not being an expert on SSIS, this write up might be a bit bias towards BizTalk but hopefully will provide some useful comparisons or insights.

What is SQL Integration Services (SSIS)?
SSIS is a replacement for Data Transformation Service (DTS) and provides tools to build data integration solutions, including extraction, transformation, and load (ETL) for data warehousing. SSIS can be used to build solutions to transfer/copy data between multiple different data sources (Database table, Flat File, Excel, etc) using a variety of protocols (SQL, FTP, SOAP, etc) allowing for transformation/cleaning/aggregation between those data sources. SSIS also provides a .NET Application Programming Interface (API) and can be invoked from .NET code.

What is BizTalk Server?
BizTalk is an Enterprise Integration (EAI) and Business Process Management (BPM) Server. It provides tools to create both long-running and transactional business processes allowing for interaction with multiple platforms through adapters supporting multiple protocols (SQL, File, SOAP, HTTP, POP3, etc) and applications (SAP, Siebel, JDE, etc). BizTalk includes a Rule Engine allowing for real-time execution of business rules, a XML transformation engine, real-time monitoring framework for better visibility of business processes and more.

Overlapping features of BizTalk and SSIS

• Integration
  SSIS provides integration on the database/physical file level and some support for integration with a business process e.g. through SOAP (although this is not typical usage of the product). It’s typically used to execute batch processes without specific or changing business rules. A typical integration scenario for SSIS is Database to Database integration or File to Database (or vice versa) batch process that is scheduled to run at a given time.
  
  BizTalk provides integration on the business process level but also support for database level integration. BizTalk can receive data from both databases and files, execute a set of discrete business step and write data back to a file of database. BizTalk’s strength lies in real-time data, orchestration of business processes and ability to connect many different systems and applications. BizTalk adapters also have features such as automatic retries and service windows. A typical integration scenario for BizTalk is a multi-step business process requiring specific business rules to be executed and multiple systems to be interacted with e.g. an order and credit-card approval process through a 3^rd party web service.
  
• Transformation
  SSIS support transformation between many formats, e.g. comma separated files, Excel files, and database table but only offers very limited support for XML. XML is supported only as a data type (e.g. to be inserted into a table) but XML cannot be transformed to Flat File format without addition of custom code.
  
  BizTalk supports mainly transformations of XML to XML. The reason for this is that all data that goes through the BizTalk engine need to be converted to XML. BizTalk has support for flat files as an XML schema but the mapping between transformation formats is done on the XML level. Custom pipeline components can be created to support transformation to/from other formats.
  
• Messaging
  Service Broker (part of SSIS) is First-In-First-Out (FIFO) highly-scalable queue used for asynchronous and reliable processing. As it’s a FIFO no parallel processing is available which decreases throughput. The Service Broker doesn’t support subscriptions and needs to be explicitly invoked (e.g. via SOAP, Stored Procedure or through SSIS directly). There is no support for automatic retries or service windows. The typical scenario for Service Broker is to handle low priority asynchronous call from an SSIS process such as sending emails without holding up the existing process.
  
  BizTalk features a powerful publish and subscribe messaging system. New message subscribers can be added with ease (after deployment) and content-based routing is easy to implement. It supports both parallel and serial (FIFO) processing. All adapters also support automatic retries and service windows. All messages are stored transactionally in the MessageBox database.

Scenarios

• File Order Batch
  A company’s proprietary order system provides a flat file of the days orders exported from the Mainframe. The order batch data needs to be inserted into a ODBC database table.
   
  • SSIS – Everything can be done out of the box.
    
  • BizTalk – Everything can be done out of the box but SSIS is would most likely be more suitable (unless the file needed to be processed at the time it was created rather than at a scheduled time).

• Simple Order Exporter
  At a given interval, retrieve XML order data from a database, transform data, and send transformed data to a web service (which provides a thin wrapper of a database). Receive web service reply and handle error conditions based on reply or errors.
  
  
  • SSIS – Everything except the XML transformation and the handling of reply can be done without custom code. If the format could be changed and the service boundary removed (talking directly to underlying database) SSIS would be a good choice.
    
  • BizTalk – Everything can be done out of the box.
    
    
• Order Batch and Credit Card Validation
  At a given interval, receive order batch (as XML), perform rules on each discrete order (and react to rule outcome), call 3^rd Party web service for order and credit card verification and run rules on the returned result (and react to rule outcome).
  
  
  • SSIS – Everything except running business rules can be done without custom code. Every time a business rule changes (if we assume it’s been implemented in .NET code) the .code has to be updated, recompiled and deployed. Processing can only be done in serial, as one batch.
    
  • BizTalk – Everything can be done out of the box with little custom-code both in parallel (by splitting the batch and processing each order individually) and in serial, as one batch.

Summary

SSIS strength lies in bulk data processing on the database/file level. SSIS is a great tool for implementing ETL solutions but not a good tool for implementing business processes on. If data needs to be interpreted, business rules needs to be evaluated and multiple steps are involved BizTalk is most likely a better choice.

BizTalk strength lies in supporting complex processes on the business level still with good support for data level integration. To use BizTalk for database to database integration without/or with few business rules is an over-kill solution and SSIS would most likely be a better fit.

If you need to deploy a BizTalk solution in an automated fashion using MSBuild take a look at the following:

Microsoft SDC Tasks - http://www.codeplex.com/sdctasks/

Contains MSBuild tasks for BizTalk deployment tasks like:

• Removing/Adding BizTalk Applications
• Starting/Stopping Application
• Terminating Host Instances
• And whole heap of other goodies..

BizTalk Build Generator - http://www.codeplex.com/BizTalkMsBuildTool/

• Generates a complete set of MSBuild scripts for your BizTalk Application. Very easy and quick to use and gives a very good basis for your own customised build scripts.

Combining these two tools give you a lot of flexibility and clean build scripts if you want /need to stick with MSBuild.

The only things that is missing in my opinion is a task to Restarting the BizTalk Host Instance. To do this manually (using NET START/STOP) through MSBuild you can use the following target:

<PropetyGroup>
   <BizTalkServiceName>BTSSvc$BizTalkServerApplication</BizTalkServiceName>  
</PropertyGroup>

<Target Name="RestartBizTalkHostService">
   <Message Text="Stopping the BizTalk Service.." />
   <Exec Command="NET STOP $(BizTalkServiceName)" />

   <Message Text="Starting the BizTalk Service.." />
   <Exec Command="NET START $(BizTalkServiceName)" />
</Target>

You can obtain the value of BizTalkServiceName from the Service Console.

For you who know me personally and would like to hear of my adventures overseas I have created a new blog explicitly for this purpose on:

http://bamontour.blogspot.com

I have lately been playing around using WMI though .NET (System.Management namespace). The reason for this is that I’m developing a little BizTalk admin tool and I’m using a fair bit of WMI interop as well as ExplorerOM.

I never realised the power of WMI until I started doing this. WMI allows you to do many things among control hardware, hook in to OS events and all MS Server products exposes functionality through WMI (e.g. BizTalk). It basically brings you closer to the platform.

WMI programming is fairly simple to do through managed code these days, and it’s made even simpler through this free MS Tool: WMI Code Creator. This tool lists all WMI Classes, events and even generates code in C#/VB.NET or VB Script.

Nice Nice!

The first BTS 2006 has finally hit the shelves!! It seems alot of author have decided to hold off releasing their books until BizTalk Server 2006 R2 has been released but here is a book that takes the plunge.

BizTalk 2006 Recipes: A Problem-Solution Approach is based on the experiences of many of the most prominent experts in the field. It features over 170 problem-solving recipes for BizTalk developers and administrators.

BizTalk Server 2006 builds on the heritage and core architecture of BizTalk Server 2004, leading to a powerful tool that encompasses the latest Microsoft technologies and industry standards for automating and managing business processes. BizTalk Server 2006 adds incremental value to BizTalk 2004 by improving administration, deployment, and other key areas of the product.

Book Web Site

Suppose you came across a flat file format that describes a header, a footer and a collection of different types of unrelated object (e.g., CAT, DOG, CAR, HOUSE, SPACESHIP). Each object have a different stucture and it's own TAG identifier.

The format could look like this:
<Header Comment>, <Sequence Number>, <Date of Transmission>; <List of objects>;<Total Number of objects>,<Sequence Number><CR LF>

Each object is a completely different structure and a has unique tag identifier e.g:
<Tag Identifier> <List of Fields separated by , >;

And example of the example of data looking like this could be (colour coded for readablity):

A Header Comment, 12345678910, 2006-09-19;CAT Garfield,5 Years,50KG;DOG Odie,Drowles Everywhere,Loves Everything,10KG,2 Years;CAR Mazda 323,Marcus Rosen,123 QYW,Red,1999;CAR Honda Accord,Belinda Something,567 QWE,Silver,1997;HOUSE 10 Somewhere,Some Town,Some Country,12345;SPACESHIP The Falcon,200 Lightyears,Luke Skywalker;CAT Tiffany,13 Years,12KG;CAR BMW 318i,Scott Martin,412 YQW,Metallic Blue,2006;8, 12345678910;

Can you create a Flat File Schema that supports multiple repeating TAG identifiers in the BizTalk Schema Editor?

YES you can but it's not really straight forward, hence this post!!

Unfortunatly I lost the images that were previously uploaded here but the example files are still avaliable for download.

Download Example

Here is a great older post by Brian Loesgen trying to weed out the confusion about when to use Windows Workflow Foundation versus Biztalk Server 2004/2006.

http://geekswithblogs.net/bloesgen/archive/2005/10/09/56481.aspx

Sreedhar Pelluru, Programmer Writer on the BizTalk Server team has a great post (with code examples) about changes in the Business Rules engine between BizTalk 2004 and 2006.

Change Summary:

• Support for generic types and generic methods
• Support for nullable types
• Accessing nested members of a class
• Type Casting support
• Invoking Static Members of a Class
• Overriding registry key setting with application configuration file.
• GetDeploymentDriver method added to Configuration class
• Clear method added to the Policy class
• IFactRemover interface added.
• SqlTimeOut registry key added.
• Arithmetic and logical operators support double promotion

On the 30th of August I had the pleasure of presenting to the Brisbane BizTalk User Group - BrizTalk. The presentation covered a solution I implemented last year using DSML (Directory Service Markup Language) and BizTalk Server 2004 to move users between a mainframe system and MS Active Directory.

Unfortunaly the Powerpoint presentation has been lost. :( I will post it here again if it surfaces.

I had the pleasure of attending the BizTalk 2006 Deep Dive Course in Seattle, Washington.
This course is (without a doubt) one of the worlds best advanced traning courses in BizTalk Server.

The course covered the following (and more):

- BizTalk Performance (deployment and orchestration optimisation)
- Security
- Advanced Mapping
- Advanced Messenging
- Integration Patterns
- Advanced Flat-File Processing
- Advanced Orchestration Design

I absolutely recommend this course to anyone wishing to further their BizTalk skills!

I sat the BizTalk 2006 Beta (71-235) exam in the beginning of June and the results were released last week.
I PASSED!

Anyway, compared to the BizTalk 2004 exam this was nothing. The questions were very high-level and a person without a lot of experience could have passed it. The most focus (from memory) was on BAM but questions regarding the Business Rules engine, Persistence Points in orchestrations and general orchestration knowleadge was also covered. The study guide ( which I spent about 1 1/2 days with) can be found here and matched quite well the exam content.

Next on my list are the MCDP: Enterprise Developer UPGRADE exams.

I have for the last couple of days been playing with K2.NET, which is an excellent product for playing the human workflow part in a BizTalk implemention, rather the awful Human Workflow Services.... :-(

As I was trying a couple of different things, I got to a stage were I wanted to delete some of my existing deployed (K2 uses the word "exported") processes, but I could not for the life of me work out how to do it. It's not in the HELP file, not in the study notes, etc..

Finally I found this post on the K2.NET Forums that discussses the wProcDelete stored procedure in the K2 transaction database...

It isn't a very pretty, intuitive or even user-friendly solution but it does work....

Tried for the 1st time to install BizTalk 2006 myself today on a Virtual PC Image, but had problems downloading and finding the pre-requisite redistributable files (BtsRedist.cab) required for the install.

Thanks to Grant Holliday of Readify it all worked out, see his post here:
http://www.holliday.com.au/blog/biztalk-2006-pre-requisite-cab-files.html

The BizTalk Product team has released the product documentation in PDF format (> 8000 Pages)!! The package contains the following PDF files:

• Getting Started (506 pages)
• Planning and Architecture (1,056 pages)
• Security and Protection (60 pages)
• Development (2,098 pages)
• Deployment (149 pages)
• Operations (1,950 pages)
• Business Process Management (703 pages)
• Troubleshooting (133 pages)
• Technical Reference (2,100 pages)

New Features of BizTalk Server 2006

Kris Horrocks introduces some of the new features of BizTalk Server 2006. See demos of the all new BizTalk Administration Console and Group Hub Page, as well as technical demonstrations of new core engine features like recoverable interchanges and end-to-end ordered message delivery.

http://msdn.microsoft.com/msdntv/episode.aspx?xml=episodes/en/20060309BizTalkKH/manifest.xml

Microsoft has released a whole heap web casts for SQL Server 2005:

http://www.microsoft.com/sql/eval/webcast.mspx?wt.mc_id=SQL.AD.01072

Are you programming in VB.NET 2.0?

Are you missing the refactoring function that C# 2.0 offers in Visual Studio 2005?

I know that I was really looking forward to this function when finally moving to VS2005 and was very disappointed when the VS development team was forced to remove it from the first realease of VS 2005 due to time constraints.

WELL, Developer Express Inc has now released a free add-on for VB.NET 2.0 that should offer similar functionality to the C# one.

Get it from here today:
http://msdn.microsoft.com/vbasic/downloads/tools/refactor/

After about a month’s preparation I finally sat and passed the 074-135 (Developing E-Business Solutions Using Microsoft BizTalk Server 2004) with a final exam score of 887 (pass is 700). It's a hard exam in that the preparation required is quite extensive. As with any MS partner exams there are no practice exams (e.g. Trancenders) available and all you have to go on is the study guide. If you intend to sit this exam, I highly recommend lots of PRACTICAL knowledge regarding the platform (as I actually doing versus only reading about it).

These are the source that I used:

BizTalk 2004 Server Unleashed
BizTalk 2004 Whitepapers
Microsoft MSDN Articles
MOC 2157 – Development
MOC 2158 - Administrator
All SDK samples (Refresh)

I do believe in this case the study guide matched pretty well what the exam did cover.

Now I only have the BizTalk 2006 exam to look forward to, which is yet to be released. ;-)