Scheduling and optimising XML pipeline processing

Abstract

This thesis describes PropelXbi - an implementation of the XPipe paradigm - then investigates and critically assesses relevant techniques which have the potential for streamlining PropelXbi’s performance and, finally, it presents and tests improvements in PropelXbi which are achieved by implementing a number of devised enhancements. XPipe is a paradigm for processing a great number o f very large XML documents in an efficient way. PropelXbi is a commercial implementation of the XPipe paradigm based on JMS and J2EE architecture. Relevant topics we have investigated include architectures and enhancement techniques used in parallel processing, Jackson Inversion, TupleSpaces, Project JXTA and Grid computing technologies. We have implemented a J2SE-based compact version of PropelXbi runtime (compiled pipelines) and a Grid-based distributed version of PropelXbi. Tests showed that the compact version of PropelXbi runtime achieves significantly better performance than original J2EE version. Tests also showed, that distributed processing can be used for streamlining PropelXbi’s performance and that the distributed version follows the same laws as other standard parallel processing systems. This thesis identifies potential enhancements from different areas o f computing which can be used not only for streamlining PropelXbi, but also for any other similar large-scale document processing system, and demonstrates that they can be efficiently utilised.