Smart Port Platform monitoring
Prodevelop case study is a smart platform in charge of monitoring the activities of a port in real time, through the analysis of data coming from sensors (IoT) and information systems (legacy and external systems). This platform is a data intensive system that receives and analyses thousands of data per second using big data technologies running in virtual machines of containers on premise or on cloud infrastructures.
The purpose of smart port systems is to monitor the operations of the ports and analyze all the information received to detect problems and optimize the use of resources (cranes, trucks, employees, etc.). The current problem when developing these systems is the complexity of the architecture with more than a dozen nodes working in unison, the large amount of information to process per second, and the necessary customization for each port and terminal. This smart platform has been developed using different big data technologies (NoSQL databases, message brokers, Dashboards, Complex Event Processors, Open data, AI algorithms, ELT “Extract Load Transform”).
Thanks to the AIDOaRt Project, we expect to help developers to more easily find errors in the predefined information flows, as it is currently difficult to track the flow that follows the information, as well as identify the node/process that is producing errors, bottlenecks, and so on. Both the detection of anomalies and the proposed visualization tools will be of great help to develop more robust products and in less time.
On the other hand, the sizing and deployment of these systems are complicated tasks because of the peculiarity and different requirements imposed by different installations. In fact, a typical problem is the correct dimensioning of an infrastructure for given demands, avoiding over-dimensioning and, thus, higher monthly operational costs. In this scenario, having a system that allows the deployment of the infrastructure in an automatic way would be a plus, since there are so many nodes and software components involved and it is not easy to carry out the deployment. Moreover, the number of nodes and the configuration and selection of software components make it a non-trivial task.
Once the system is deployed, its monitoring is another aspect to control, since both the nodes and the software components can have temporary or permanent failures and the system should be able to automatically detect and correct them, being able to create new nodes and migrate or run software components between nodes.
Finally, monitoring the flow of data and ensuring that the information is processed correctly without loss of data is another key point. This aspect affects the system recommendations that can vary considerably if not all the data is considered and in the correct order, resulting in a great negative impact on productivity.