Data Sync Functionality
The core focus of this release was to enhance DBIntegrate to take advantage of “Change Data Capture” or CDC, which is the processing of taking events in our legacy applications e.g. ISAM/COBOL, or RDBMS and applying associated actions into another database, for example SQL Server. We have termed this functionality ‘Data Synchronization’ (Data Sync).

Our Data Synchronization graphical interface for defining jobs has retained the look and feel of the DBIntegrate Data Mapper, so there will be minimal retraining to get up and running with your job if you are already familiar with product.
The use of drag ‘n’ drop functionality and easy to use interfaces makes first time use straightforward and painless:

Data Sync wasn’t the only work and improvement we carried out in this release, we are always working to refine our code and carry out code reviews on a weekly basis to help bring you the best product available. We constantly work on improving DBIntegrate in the background, but some of the other features you might notice that this release brings to you include:

Support for access to Fixed-width text files
This allows you to import your text file, select column widths and assign column headers.

Data De-duplication Results Viewer
We have created a deduplication “results” viewer, this enables users to identify which records have been matched against each other.  This audit trail allows users to view and refine the matching or merging processing steps.

NEW Processing Functions
We are also continuously identifying and adding new functions to DBIntegrate that have not only been useful for us, but we think will help you as well. The new functions enable for greater manipulation of your data without the need to code and create your own custom functions.

Here is a selection of some of our new favourite functions:

• DBI_CHECK_EMAIL_FORMAT: Identifies if e-mails are in a valid format, and returns a Boolean result.
• DBI_REMOVE_EXTRA_SPACES: Removes additional spaces within a string
• DBI_SUBSTITUTE_STRING: Lets users enter in a string they wish to switch for another
• DBI_REMOVE_PUNCTUATION: Removes any punctuation from a string
• DBI_SENDEMAIL: Send e-mails straight from the Data Sync or Data Mapper runtime

NEW DBIntegrate Variable
We have added in a new variable; s_action_err. If used during runtime this identifies if a record has failed on insert/update or delete to a target source. It can be called within IF conditions to apply further processing or user notification, such as in conjunction with DBI_SENDEMAIL to send an e-mail to a specified admin team (for example) to notify them of erroneous data.

Please find the release notice here where you can view further information on new features and issues that have been resolved since our last release.
If you would like a free-demo of DBIntegrate please visit our website for access here

Please note: the free-demo version has reduced functionality; Data De-duplication & Data Sync are not available for free use.

[FG4.RUNW]

OemToAnsi=0

We’ve done a huge amount of work for this release although much of it is ‘under the covers’ providing much more flexibility, improved performance and reliability for all data access going forward.

Here are the highlights:

• Improved Data Access.
  All data is accessed directly in order to improve performance. CSV format files are accessed through a native API to provide speed and flexibility. In future releases we will build on the improvements in this area.
• Data Source Definition Wizards
  We’ve worked on the Data Source Wizards to make it as easy as possible to define a data source. In most cases it takes just a few seconds.
• Logging
  We’ve standardized the logging for all the different types of processing. This will also enable access to logging from user defined functions. Logging can be configured in the Project Designer for every project type.
• Data Merge and De-duplication
  Data Merge and De-duplication jobs can now be run in separate steps. This will really help enable an iterative and collaborative approach to data integration project development.
• Processing Functions
  We’ve added more new processing functions, including functions that make use of a built-in database. For example, we have a function that gets the ISO country code for any specified country name.

For more information, please look at the release notice.

Contact customer services for product download or go direct to our FTP site.

Plans for the next DBIntegrate release will be posted very soon.

This version includes improved support for the new alternative query mechanism. This new functionality originally introduced in version 3.40 provides application developers with a means of defining alternative SQL queries to override the default queries generated by the DBPronto server.

Alternative queries can be more efficient and potentially faster than those generated by DBPronto since they can take advantage of the application developers in-depth knowledge of their system.

Alternative queries can include joins that enable multiple tables to be read using a single query. When using such joins care must be taken, as if too complex they may have a detrimental affect on performance.

The alternative query mechanism requires only minimal program changes since the queries and how they relate to I/O statements are defined in metadata files. The mechanism currently only supports the READ and REWRITE statements on COBOL index files.

Please refer to the DBPronto documentation for more information.

Have fun.

ftp://ftp2.transoft.com/pub/CDextras/UBL/2011-03-15UBLStudioWebinar.wmv

Go to www.transoft.com/webinars to register today!

DBPronto server Table Caching

DBPronto can be made to use table caches for improving the performance of table read operations, especially when performing random read operations. The DBPronto server will automatically update a table cache when performing updating operations such as OPEN OUTPUT, WRITE, REWRITE and DELETE.

Please refer to the exciting, newly overhauled DBPronto documentation for more information.

In the near future we plan to enhance this functionality so that it can be used with the client block reading mechanism so as to reduce the DBPronto client/server communication and provide further performance benefit.

Identifying owners of locks from the DBPronto Administrator

The DBPronto Administrator has been enhanced to display the PID and user name of users connected to the DBPronto server. With the session ID and the list of record locks, this information may be used to identify the users who have particular records locked.

Opening over 250 files

The DBPronto client libraries have been enhanced so that the maximum number of file’s that can be opened at one time by a process is now configurable. Previously this value was limited to 250.

New API calls to fetch multiple rows using an SQL query

The problem with the DBPronto SQLCMD API routine is that when used with SQL queries, it may only be used with those that return a single row. To overcome this restriction we have introduced the SQLQUERY, SQLQUERY_FETCH and SQLQUERY_CLOSE API routines.

This functionality may prove useful in the few cases where existing program logic doesn’t work well with DBPronto and standard file I/O, and can better be expressed with alternative SQL coding. An example of this are batch jobs where large volumes of data are read but performance is critical.

In the near future we plan to enhance this functionality so that it can be used with the client block reading mechanism .

Table Maintenance routines

We have enhanced the DBPronto client libraries with new routines to enable application programs to instruct the DBPronto server to globally close and reopen tables, and perform metadata refresh operations.

The close and reopen routines may be used to prevent applications from accessing tables whilst for any reason they have to be taken off-line. The reopen routine may also be used to create tables that do not already exist.

Using SET_WHERE with the DBPPP

Enhanced the DBPronto pre-processor so that it will recognise a new DBP-SET-WHERE statement that will be converted to a call to the DBPPP_SET_WHERE routine. This enhancement was especially useful for RM COBOL application that wish to use the SET_WHERE functionality which may be used to restrict rows selected by queries.

The metadata creator to sort alternate keys by byte order instead of SELECT order.

We have enhanced the metadata creator to define alternate keys by byte order instead of SELECT statement order. This behaviour is now the default when generating metadata for use with ACU but can also be enabled and disabled using specific command option lines options.

One of the big enhancements is the implementation of configurable table caching.

To give you an indication of the benefits, the table below shows the time taken by a program using U/FOS data access to read 8000 records from a  U/FOS database compared with the same program reading from a SQL Server database via DBPronto with a variety of configuration options:

This demonstrates the fact that DBPronto performance can be improved using a range of configuration options which are designed to be applicable in different situations depending on the data structures used and the type of processing.

Another option is to use DBSQLCMD and DBFETCH commands to access the data using specific SQL. This can be very helpful in the very few cases where existing program logic doesn’t map well onto RDBMS data access and the problem is difficult to solve using DBPronto configuration alone.

It is now much easier to navigate and includes a new, easy-to-use ‘Getting Started’ section.

Please download DBPronto.chm from ftp://ftp2.transoft.com/pub/DBPextras/

First, install the U/SQL Server and the U/SQL Client and get them both licensed. Note that you’ll need a gateway client license in order to install the U/SQL client on Windows 2008 Server.

Install, license and start the U/SQL Server

The next step is to check that you can access the demo data from Win U/SQLi :

Select books32.udd, a 32 bit ODBC DSN

Test data access

Now, to access the data from 64 bit SQL Server you need to create a 64 bit ODBC DSN.

Start the Windows ODBC Administrator from Administrative Tools – this is the 64 bit Administrator so you won’t see the 32 bit DSN for books32.udd listed.

Create a new DSN and select the 64 bit Transoft ODBC driver

Create a new DSN named books64.udd

Now add books64.udd as a new data source in the U/SQL Manager:

Add books64.udd in U/SQL Manager

The next step is to start up SQL Server Managment Studio and create a new Linked Server:

Create a new Linked Server named BOOKS64

Select the Microsoft OLE DB Provider for ODBC Drivers and specify the 64 bit DSN

Now you can run a SELECT query to read the U/SQL demo data:

Example using 'select * from openquery' syntax

Example using fully qualified (four part) table names

However, if you attempt to INSERT data, SQL Server returns an error:

INSERT gives error 'The requested operation could not be performed because the OLE DB provider 'MSDASQL' does not support the required transaction interface.'

The solution is to check  ‘Non transacted updates’ in the  MSDASQL provider options as follows:

Check 'Non transacted updates'

Now, create a new Linked Server, BOOKS64_1, and try the INSERT again:

INSERT into BOOKS64_1 works correctly

NOTE: INSERT into BOOKS64 still fails because the provider options are set when the Linked Server is created.