Data-Intensive Engineering

References

Cite

Share

Abstract

Part I of this book discussed the evolving landscape of data-intensive computing and the challenges that organizations and individuals face in extracting knowledge and, eventually, value from data. Part II delved deeper into the analysis processes needed to perform data-oriented computations. These processes included not just analysis techniques, patterns, and approaches to deal with widely distributed and unstructured data, but also strategies for adapting the higher level development processes to deal with dynamic and changing data sources. Part III, this part, is targeted at technical experts, mostly data-analysis experts and data-intensive engineers, expecting to develop components or complex applications in DISPEL and those considering developing platforms supporting the data-intensive architecture outlined here. There may be some domain experts, who will find this discussion interesting, but like the automobile it is not necessary to understand the engine to drive the car; this part is for those that want to understand and work on the data-intensive platform. While most of the focus so far has been on knowledge discovery, the architecture outlined is of a general nature and should be considered by practitioners across a wide range of (data-centric) specialities, for example the programming paradigms such as Hadoop [1], mapreduce [2], and related technologies. We expect data-analysts from many disciplines to make effective use of the data-intensive platform and to contribute its expanding capabilities, including general technologies of data-intensive distributed computing, such