Introduction to File Locations Monitoring

Introduction

File Locations Monitoring is one of the new Monitoring capabilities introduced in BizTalk360 v8.4. From our customers, we received many requests to introduce the ability to monitor Folder Level into BizTalk360. We are bringing a feature to monitor file count for File Locations (File, FTP and SFTP), configured in BizTalk receive locations and send ports.

Need for File Locations Monitoring

Messaging or data exchange between business can be done in various ways. Frequent data communication process is done through for example File and FTP with XML or EDI as the message format. From BizTalk 2013 onwards, SFTP is also included in BizTalk Server which was available as Out-of-box feature in prior versions. Even though BizTalk works seamlessly with File Adapters, it has some known issues which occur due to incorrect configurations. The BizTalk File Location Adapters (File, FTP, SFTP) fail to perform the operations on following scenarios:

The File receive adapter cannot access the receive location on the file system or network share because the specified path does not exist. For a network share, the File receive adapter disables the receive location after all retry attempts have been exhausted.

The File receive adapter cannot access the receive location on the file system or network share because the account used by the associated host instance does not have read-write permission for that location. For a network share, the File receive adapter disables the receive location after all retry attempts have been exhausted

Files with names longer than 256 characters are encountered in the receive location

To resolve the above issues, we need to ensure that the specified path or share exists and the account used as the Logon should have read-write access. Additional to this, if you configure schedule/service window for your receive locations, messages will be accepted only during that time window, all other times BizTalk wont pick up messages. Any violation to this scenario also needs to be monitored. We often experience that organizations facing these kinds of challenges used custom solutions for this kind of monitoring. To overcome this, BizTalk360 added the File Locations Monitoring capabilities out of the box.

File Locations (File, FTP, SFTP)

In BizTalk360 v8.4, we are introducing support to monitor the File, FTP and SFTP servers under File Location Monitoring Section. File Location Monitoring will list all the locations configured in the BizTalk Artifacts (Send Ports and Receive Locations) for the Transport Types (File, FTP, SFTP) respectively, which helps users to monitor all the File Locations mapped with Receive Locations/Send Ports.

File Monitoring

In BizTalk360 File Monitoring Configuration contains three sections: Basic Information, Authentication, and File Monitoring Configurations.

The Basic Information Section contains Folder Location and File Mask configured in BizTalk.

The Authentication section is Optional. By default, authentication could be processed by the BizTalk360 Monitoring Service account, when credentials are not given.

In the File Configurations Section, we can configure the Thresholds with the metric File Count to monitor

When a File Location is in the Orphaned State, BizTalk360 would let the users know about the cause of the failure on hovering the warning icon

Messaging or data exchange between business can be done in various ways. Frequent data communication process is done through for example File and FTP with XML or EDI as the message format. From BizTalk 2013 onwards, SFTP is also included in BizTalk Server which was available as Out-of-box feature in prior versions. Even though BizTalk works seamlessly with File Adapters, it has some known issues which occur due to incorrect configurations. The BizTalk File Location Adapters (File, FTP, SFTP) fail to perform the operations on following scenarios:

The File receive adapter cannot access the receive location on the file system or network share because the specified path does not exist. For a network share, the File receive adapter disables the receive location after all retry attempts have been exhausted.

SQL Server availability monitoring

Server availability monitoring provides the ability to monitor failover SQL clusters and standalone SQL Servers using the protocols Ping or Telnet. This feature answers the question Are the BizTalk/SQL Servers up and running?. BizTalk administrators can choose the option when to receive the alert; either if one of the Servers has gone down or only when all the servers were down.

Azure Service Bus Topics Monitoring

Users can effectively monitor Service Bus Topics and their subscription by simply configuring the Namespace connection string and Topic Name. Once the topic is configured BizTalk360 will detect and list the topics details such as its current state, message size, and subscription details. With this user can monitor the state of the topic and by configuring threshold rules you can monitor other topics and subscription metrics.

State-based monitoring The state can be monitored by configuring the expected state as Active /disabled /Send Disabled. BizTalk360 triggers an alert If the current state of the Topic/Subscription is not as same configured Expected state. Users can also set up Autocorrect for this, where the BizTalk360 system will automatically heal (change the state) if the expected state is not equal to the current state.

BHM Profile Management

BizTalk360 has already an integration with BHM. This integration enables the BizTalk360 user to schedule BHM and view the output of the different runs of the tool directly from within BizTalk360. You can also manually run BHM, Schedule BHM runs, Run BHM profiles, view and monitor the BHM runs.

You can manage and monitor only the default profile in the previous versions. From this version, we have extended the scope to support multiple profiles.

Manage BHM Profiles The detailed reports of multiple BHM profiles can be viewed in BizTalk360 operations modules and the profile can be analysed and run by clicking the Run BHM option, which runs all the profiles and generates the report.

Monitoring multiple BHM profiles The Schedule can be configured to run the selected profile on a specific time duration. More than one profile can be monitored by configuring the threshold rules and mapping that to alarm. BizTalk360 checks for any threshold violation based on the rules configured and the latest report generated and if there is any violation, the same will be notified through an alert.

FTP Monitoring

The FTP Configuration UI is categorized into three sections: FTP Details, Firewall Details and FTP Monitoring Configurations. Monitor FTP/SFTP/FTPS sites easier with an FTP Monitoring tool like BizTalk360. It offers monitoring of Receive Locations and Send Ports.

The FTP Details Section contains the details about the FTP Location, Authentication, and SSL

The Firewall Details contains the configurations to connect FTP Server through a Firewall

In the FTP Monitoring Config section, we can configure the monitor with Threshold Conditions for the metric File Count

SFTP Monitoring

The SFTP Monitor Tab in BizTalk360 lists the SFTP Locations which are configured in BizTalk. It contains four sections:

SSH Server Section has the details about the SFTP Location

The Proxy Details Section is optional to connect SFTP Server behind a firewall

Conclusion

BizTalk360 brings the File Locations Monitoring with the ability to monitor the File Count. In the future, we will be adding support to monitor Folder Size and Access permissions.