System integration

It's no secret that "everything has been done before us." Remained just a little "gather fragments" to solve the task. And then it turns out that integrating disjoint parts is not often more difficult than writing them. Why does this happen? What can be done about this?

All programmers like to make systems from scratch, when thought can freely invent any forms and means when it is possible to make decisions without looking at the legas. Of course, a system designed entirely, provided that it was made by a specialist, always looks monolithic and pleases the eye. But after all, our reality is flowing and with time, any conceptual idyll is violated in the course of business development, process changes, mergers or mergers, the introduction of new systems, the change of hardware or software platforms, and even legislation.

Who supported and implemented the systems, and even more so, engaged in refinement, reengineering and integration, he knows that more than two thirds of all efforts in IT (attention, time and money) go to the "gluing" of incompatible and attempts to "marry" the modules written different people, at different times, in different languages ​​and technologies, under different platforms.

Factors affecting integration

• Accelerating processes. The development of an organization requires more and more frequent changes in data structures, business processes, not to mention the design and user interface, which is just constantly changing. Here, precisely in such dynamic areas, where "variability" is the very essence and nature of the system, the task of integration is aggravated and turned into a serious problem.
• Distribution. Organizations are becoming more and more complex, and logical, organizational and geographical dispersion appears.
• Heterogeneity. In a major project, there is almost no way to stick to platforms and tools from a single vendor, so you have to take into account and support the features of several platforms.
• Heredity. Impossibility to completely abandon legacy systems, obsolete technologies, old hardware, which, by the way, sometimes give quite good indicators for reliability and performance, but they do not contribute to integration as much.
• Chaoticity. It is not always possible to fully formalize, specify and structure the data, and part of the model remains "weakly linked," not capable or poorly capable of machine processing, analysis, indexing, and counting.
• Conditionality. Unfortunately, information systems are limited not only by the technical framework, but also by the habits of people (which are difficult to retrain), the peculiarities of legislation (which is simply not ready for the appearance of such systems), and many other factors not dependent on the developers.
• Interactivity. The consumer is constantly increasing his expectations about the speed of the system's response, the speed and speed of delivery of information. Most processes tend to perform in real time.
• Mobility. The user of the systems began to move faster, and interaction with it is carried out through public communication channels in transport, at home and on the street, in public places and everywhere.
• Security. While the data was stored on the media inside the protected area, nobody was particularly worried about encryption, but now the network packets fly in the air and this can not be left without attention.
• High loading. The complexity of integration is influenced by: the number of users in the system, the intensity of the flow of data, the volumes of data and the resource intensity of the calculations.
• Continuity of the work cycle. Integration and upgrade systems should almost always be carried out without stopping their operation, smoothly, gradually and imperceptibly for the organization and its customers.
• Intersystem integration. Docking tasks are not limited to the organization, more and more often it is necessary to integrate with partners, clients, suppliers, contractors and even state structures.

Parameters that are responsible for the complexity of integration and will offer options to minimize the negative influence of these parameters

• The conceptual difference is based on the fact that the developers of different systems initially made different decisions, assumptions and assumptions that conceptually do not fit together. It is solved by introducing another layer of abstraction, which conceptually does not contradict both approaches. In this case, there are two variants of realization: (a) when the resulting system becomes centralized, and two or more integrable systems become subsystems and (b) when we use the architecture of a broker (an intermediary that is not a center), while the systems remain independent, and the broker provides a layer between them.
• The technological difference is when we have incompatible data exchange formats, communication protocols and interfaces. Solves by writing envelopes, interlayers, brokers and other lotions, not quite beautiful, but quite reliable.
• Incompatibility of licenses. I will not dwell on this in more detail, since I am not an expert in this matter, but the decision can be individual in each case, at the organizational level.

The general problem is as follows: it is necessary to integrate N information systems characterized by the factors described above, with the minimization of the number of interlayers, converters, brokers and interfaces between them. If the problem is solved in the forehead, then N (N-1) / 2 constraints will exist between the N systems, that is, for two-way interaction of the N (N-1) interfaces. If we take into account that we can understand everything from the web interface to the offline process, starting, for example, once a day, and doing a number of complex operations for database synchronization (queries, processing, exporting, FTP uploading, signaling another part of the system, so that she accepts the transferred data and completed her part of the work, and then notified the results and transferred the necessary data back to her. In general, such options will never be completely eliminated, the question is only in their competent implementation.

Arsenal of funds to solve the task

• Standardization - it is necessary and important to use as many international, state and industry standards as possible, and if some are missing and they are clearly being asked, then it is necessary to introduce corporate standards, and often it makes sense and promote them in relevant organizations for the early dissemination and popularization.
• Integration at the level of brokers. Advantages: universality - it is almost always possible to create an additional program module that will access both systems in different ways (for example, one through the database and the other through the RPC). Disadvantages: complexity, complexity, and hence the high cost of development, implementation and ownership.
• Data-level integration - that is, multiple applications can access a single database or multiple databases that are linked by replication. Advantages: low cost of integration, and when using one DBMS it becomes a very tempting solution. Disadvantages: if the database is not shielded by stored procedures and does not have the necessary integrity constraints (in the form of cascading operations and triggers), then different applications can lead the data into conflicting states. If the base is shielded and the integrity is ensured, then in this case, in parallel applications working with the same database, there will be duplicate parts of the code that perform the same or similar operations. In addition, with changes in the structure of the database, we will separately rewrite the code of all the applications that are working with it.
• Integration at the service level is a beautiful integration based on fixing interfaces and data formats from two sides and allowing quick development of intercorporate business logic. There are also disadvantages: nevertheless, there is a fixation, and if the structures or processes change, then problems and narrowly specialized, private solutions are formed.
• User-level integration is an extreme case, not automated integration, when users move data between systems through copy-paste, files, mail and other disgraces. We do not consider such methods, but they, unfortunately, are often applied at that time, until the software systems are ready, and the development of the company does not allow waiting.
• Dynamic interpretation of metainformation - we'll talk about this in a separate article.