What is Torch Migration Suite - TMS.
Torch Migration Suite is a unique Windows-based tool coupled to a step-by-step methodology dedicated to migrate Zim databases to a full-fledged SQL relational databases such as Oracle and most importantly keeping the Zim application in its entirety.
TMS is a powerful conversion facility that assures quality, consistency, and accuracy of results. Using a modern and easy-to-use graphical user interface, makes the migration process consistent and simple.
Torch is complete because it embraces from the data dictionary analysis and modification to the process of cleansing and loading data in the new SQL database.
Torch automated solution provides detailed impact analysis reports for your Zim Database environment enabling you to more effectively move forward in IT modernization. You will be able to share your data with other world class applications and tools such as report generators, business intelligence tools and much more.
Torch opens the world of applications to your company.
No more restrictions to access third party apps or to be accessed from other applications and languages. And your original Zim applications will still be running as always but with no dead locks and a better performance.
TORCH Migration Suite Components.
TORCH Migration Suite is composed of 5 main modules consolidated by the TMS Knowledge Base, a Zim database that contains all definitions and objects needed to perform the migration process. The TMS can contain information up to 999 different target databases.
Each of these modules is designed to accomplish one migration step.
Data Dictionary and Schema Analyzer (DDA)
Data Analyzer and Loader (DAPG)
Source Code Analyzer (SCA)
The Configuration Engine or TORCH Configurator, is used to set the migration parameters as the Server type, connection parameters, valid dates range, table space names for data and indexes, Zim functions mapping to server functions and others. The remaining Torch components behave accordingly to these parameter definitions.
The CE is mainly used to configure and generate the so called “intermediate database” as a mirror of the target database - usually the current production DB. This Intermediate Database is the place where all transformation will take place and will be the source of all SQL scripts and new Zim DD definitions.
The Data Dictionary and Schema Analyzer is the TORCH main module. Firstly it imports the target DD definitions, analyze them reporting any inconsistencies, allowing the user to clean and simplify possible sources of future problems.
After revised, the Zim DD is adapted to be SQL compatible and the SQL scripts to create tables, indexes, triggers and foreign keys are generated.
The Source Code Analyzer (SCA) module reads the application source programs, analyzes and reports all Zim commands that may be adapted to get the best performance and functionality.
The Source Code Analyzer Report is one of the most important tools in the fine tune process of an existing application connected the a new SQL database.
The Data Analyzer and Loader (DAL) module main purpose is to generate a foreign package (also know as a Zim Foreign Directory) that allow any Zim database to access data from Zim and SQL databases simultaneously. The generated package is called “Torch Loader database” and contains:
- Zim and SQL table definitions;
- Programs to validate the existing data and
- Programs to load data from Zim databases into SQL databases.
Using this approach it is possible to load data from virtually any Zim database into any other SQL database directly without the need of text files to unload and reload data, meaning better speed, minimum downtime and assured data integrity.
The Zim-SQL Tools is a component containing a series of table definitions and Zim utility programs that will be used during and after the database migration simplifying all the definitions manipulation. You will be able to read the SQL schema definitions, tables, triggers and so on, compare them side by side with those defined in the Zim Data Dictionary, import SQL table definitions into Zim and vice-versa and much more. You can decide, for example, from where you will maintain your application and table definitions.
TORCH MIGRATION AND CONSOLIDATION PROCESS
The migration process starts with the creation of the “Intermediate Database”, an image from the target database with no data. The entire process will be executed on this “Intermediate Database” preserving therefore the production database integrity and availability throughout the entire process.
The Zim Database Migration process using TORCH Migration Suite, consists essentially of 4 successive steps.
Phase 1 - Data Dictionary analysis, clean up and adjustment.
All metadata is analyzed and adjusted to work with SQL. Decimal places and length of numeric fields are adjusted to the SQL requirements, virtual fields are modified and many other modifications are done.
Also here are generated the new Zim Data Dictionary ready to access the SQL DBMS and all scripts needed to create the new SQL database.
The Data Dictionary Analyzer module is the responsible to bring the new data dictionaries into live during this step.
Phase 2 - Data Analysis and cleanse.
It is well known that Zim databases can shelter inconsistent and invalid data in its files, like zeroed dates, foreign keys whit no corresponding parent and so on. This is due to the absence of data validation and referential integrity controls on the database engine.
On the other hand, while preparing the migration, fields may have their attribute change, e.g. an index field may become a primary key and a “not required” fields transformed into “required”. These modifications may lead to inconsistencies within the existing data.
The Torch module “Data Analyzer” reads all tables definitions searching for candidates table fields that can contain those unacceptable values, reporting them to be fixed. Those not fixed, will have their values changed to default values during the data loader process.
Phase 3 - Application adjustment to SQL requirements.
Zim language has a series of command styles that are not supported directly within the SQL language. A typical example is a Zim ADD command that generates a “Named Set”. Programmers also use to divide long commands that deal with several tables into a series of simpler commands to create their own access strategy. These kind of programing, despite being good Zim practice is not recommended and often not allowed in SQL causing sometimes severe performance problems.
The TORCH Source Code Analyzer reads the application source programs reporting all Zim commands that should be revised to improve its performance or dua to incompatibility with the specific Server SQL language.
The report also categorize programs according to their complexity to adapt to access the SQL server, which represents a strong indicator about the time needed to adjust the entire application. The Source Analyzer also shows for some more complex commands, examples on how adjust the command to fit the SQL requirements.
Phase 4 - Target database Data Load
After the data dictionary is adjusted to SQL and the SQL database is created using the scripts generated in “step 1”, the real data may be loaded from the target Zim database into the new SQL Database. TORCH then generates a special database called the “LOADER DB” that contains both definitions, Zim and Oracle allowing to read the Zim data and write it into the SQL database.
This special database is seen as a “ foreign DB” and can be used from any other database. For example; you can access the Loader DB from you production database and load its data directly into Oracle without disturbing the production environment, speeding up the loading process several times, minimizing the “ database down time” and without the need of text files creation for unload and upload data.
During the loading processes all invalid or out of bounds data is transformed into default values set by the user during the Torch Configuration process. You can repeat the loading process as many times times as needed.