New Methods or New System?
Today, several hundred companies on IBMi are considering replacing their IT system. Over the past 10 years, some companies have left the platform by trying to standardize processes and workflows in order to adjust to a standard ERP solution on a different platform. It is our experience that this can be a costly and risky process, and that the companies can lose important competitive parameters, since there are often essential features that cannot be standardized, and which can be difficult to support on the new platform. Furthermore, it is our experience that a change in platform can prompt key employees to go elsewhere, leaving the company with a proficiency loss.
The unique thing about the IBMi platform is the fact that the customers have been able to migrate their software when upgrading their hardware, all the way back to System/36 and System/38 in the 1980’s. The platform has been backwards compatible for more than 30 years, ensuring that customers did not need to rewrite or adjust programs regularly. This has meant significant savings compared to other platforms, which cannot offer that kind of compatibility. It has also meant, however, that the developers have had no need to learn new programming languages and use different types of technical solutions. There has been no need for new competencies, and companies have been able to keep employees for longer periods than companies on other platforms have, thus ensuring continuity and integrity of business knowledge.
This situation can give both the company and the developers the impression that they have neither the competence or the technology available to meet the growing demands for integration into newer technologies and graphic interfaces. This assumption is wrong and often based on a lack of knowledge about the options on the platform. Unfortunately, some companies choose to change systems and platforms based on this incorrect assumption.
Walk the talk
As software providers, we were faced with the same considerations in 2004. We had been developing client-specific systems on the IBMi platform since the mid-1980’s. In the 1990’s, we developed a number of standard solutions for handling printing, faxing, mail, sms, and xml. We were very successful, and the product was sold both nationally and internationally. Part of the success was the fact that the solutions were based on the IBMi platform and the familiar character-based 5250 user interface, with which the users were comfortable.
With new generations of users adopting the IBMi platform came the desire for a graphic user interface. Digitalization of the society also resulted in an increasing desire to integrate companies and digital services.
It was evident that we, as software suppliers, needed to adjust to the market. We had no desire to get new developers, rewrite our solutions and announce to our customers that we could no longer support their customer-specific solutions as well as our own standard solutions on the IBMi platform.
We were bound and determined to stay on the platform. We estimated that neither the architecture or the programming language recommended by IBM in 2004 could use the competences of the developers or reuse the existing business logic. We were convinced that we would have to develop an architecture and a series of tools that would make the transformation process more attractive for both our customers and our employees.
Since 2004, we have developed an architecture and a method that respect the structure of the platform and the operating system, and which - for this reason – feels natural and almost familiar for developers, operators, and users.
We implemented the change in our company with great success and without losing employees, customers, or business logic – and as an extra bonus we were able to expand our solutions with new features that had not been technically possible prior to this. In addition, we have been able to attract newly educated IT developers under 30 years of age, reducing the average age of the company’s employees significantly and bringing in new knowledge and inspiration.
Over the past 10 years, we have developed a number of tools to expedite implementation – and we have helped more than 150 companies nationally and internationally with this protect to a lesser or greater extent.
What is the starting point?
A traditional system on the IBMi platform consists of:
· A DB2 database with physical and logical files
· Batch programs
· Interactive programs
· CL programs (a collection of command and program calls)
The DB2 database is typically described by one individual source code for each physical and logistical file. The programs are typically written in RPG or Cobol. Interactive programs interact with the user through an externally described display file, while printouts are produced through an externally or internally described printer file. All these descriptions are compiled for a machine executable object, which is also secured by the operating system against unauthorized access.
For decades, the strength in the traditional system has been these monolithic program objects that contain everything in one object and are developed for a specific work function.
In a world where workflows are changed regularly, these large program objects have become an obstacle, as they often serve one function only. If the work function needs adjusting, the entire program must be changed. Similarly, there is a tendency to develop partially redundant business logic, when new programs are developed for similar work functions. Which can mean that more programs need to be changed simultaneously.
The external description of the physical file contains information about the format of the fields as well as a very simple data validation. The format and the validation is inherited by the display file and can be further adjusted in the external description of the same. Since the simple validation is rarely adequate, the task is typically moved to the program. Furthermore, the presentation options of the display file are very limited, which means that the program also ends up containing trivial code to handle this. In an interactive program, it is not unusual that 2/3 of the program code is about validation and presentation.
In this way, the business logic has been spread over the database definition, the display file definition, and repeated in a number of variations across a number of programs.
Last and not least, the operating system ensures that the files used by the program have not changed structure since the object was compiled. For decades, this has been a valued and respected security feature in spite of the fact that even the slightest change in a file could result in having to compile a larger number of programs again.
Therefore, it is very time-consuming to do even the slightest functional changes, and it can be difficult to see the big picture of the results of a change. Therefore, companies often pass on tailoring their systems to new workflows, in time causing the system to lose the ability to support the daily work functions of the users.
We can change that.
What is the goal?
A modern system consists of:
· A SQL database with tables and indices
· A domain model describing data and functions
· Background handling of processes heavy on transactions or calculation
· A service layer based on REST/SOAP
The above points are seen again and again in most newer systems on most platforms. On the IBMi platform, the trick is to create a smooth restructuring with reuse of and respect for the code base of the company, which has been developed over decades. The code base represents very large investments and is just as important to the company as a large production plant. The code base might even be the only real documentation of the company’s formal business processes.
The operating system on the IBMi platform ensures that the current DB2 database has a complete SQL interface as well. Over the past ten years, IBM has optimized and expanded the SQL interface. Today, SQL provides access to functions and solutions that would be performance-heavy and would require significant amounts of programming code on the traditional DB2 files. The customer doesn’t have to make changes in their database definitions in order to use SQL. Physical and logistical files will automatically be considered as tables and index. In the future, however, it will be interesting for the customers’ developers to adopt some simple SQL guidelines.
Instead of repeating the definition of validation and presentation endlessly, this is collected in the Domain Model. The model is also used to define the mutual contexts of the files and perhaps supplement with smaller bits of business logic. The purpose is to centralize and remove redundant definitions. We deliver a tool for developing and maintaining the domain model.
Batch programs are usually used for background handling of processes heavy on transactions or calculation. The majority of programmers don’t need an interface or an integration into other platforms. We recommend postponing the restructuring, since apparently they will benefit from the future services. Others will be able to rewrite a number of new services
The interactive programs have a character-based user interface. They can be divided into simple and complex programs. The simple programs are often used to maintain original information and they are often connected to a smaller number of files for validation of data context. The validation in these programs is also relatively simple and rarely contains large amounts of business logic. By importing the definitions of the involved files into our Domain Model, data context, formatting, validation, presentation, and business logic can be removed from the programs. The interactive program has now been replaced by a web application with a graphic user interface, and the files can now be accessed as a service from any modern platform. In our experience, upwards of 80 % of the interactive programs are simple.
The complex interactive programs are characterized by bringing together a large amount of files, often from different system areas, for the purpose of producing new data and updating existing data. The programs typically contain more complicated validation and business logic - often with closer user interaction. Examples of complex interactive applications are order registration, product and customer registration, etc. The programs must be broken down into a number of smaller services. There will be overlap with the services of the simple interactive programs, as these are among the affected files.
It is good praxis, to create a number of small services that can then be combined as needed into a larger and more function-oriented service. There should be no overlapping services. The service architecture opens for unit testing, since it is simple to define and test constraints.
We recommend REST-based services, as they have a simpler format and are very well suited for internal use. When integrating to external partners, it may be reasonable to use SOAP because of the extra WSDL definition that follows.
What is the purpose?
After restructuring to the above architecture, the system will be divided into three levels:
Each level has a clearly defined interface. It is possible to make changes to a level without necessarily affecting other levels. For instance, a table in the database can expand by a column without affecting existing services and presentations. It is not even necessary to recompile the service programs.
In practice, this means that it is possible to make basic system extensions without shutdowns and conversion of data. It will also be possible to support new workflows simply by rebuilding existing services in a new way.
The breakdown of smaller services provides a clearer structure and makes documentation easier. Additionally, it allows the use of unit testing to check functionality. The method is widely used in modern system development and is referred to as micro-services.
Services can be individually rewritten to that programming language that best fits the task and platform at any given time. As a starting point, RPG is used in the organization, but over time more and more services will be based on things like JAVA - or whatever else the future will offer by then. Most importantly, the micro-services architecture is used and respected so the technology transaction can take place smoothly.
What do we deliver?
We deliver a complete product suite containing the necessary tools to complete the process – and thus the certainty of success:
· Web Server
· Web Applications Server
· Domain Model
· Graphic Domain Model definition designer
· Web framework containing CMS, PIM, Website, Webshop, and Webportal
· Application framework containing administration, menu system, user and security management
· Pre-compiler for RPG, Cobol, and CL.
· Tool for development of web services incl. auto-generated SOAP interface
Architecture and solution method ensures:
· that the competences and experiences of the developers and operators can be reused
· that program source and program call can be reused
· that restructuring can be done in controlled stages
· that the new solutions can run parallel with the old solutions
· that there be made no investment in or restructuring of infrastructure
· that environments can be controlled via the subsystems and library list
· that IBM’s authorization system and user management is respected
The customer’s employees are expected to be ready for change and will receive training and have appreciation of the technologies included in the delivery. Design concepts like MVVM/MVC will be used, as well as micro-services that break with the monolithic solution of the current solutions.