4.10 Parallelism for Data Explorer SMP

For completeness, the notion of module parallelism is discussed here. If you are developing visualizations or modules exclusively for use with the IBM Visualization Data Explorer running on a single-processor workstation, then these concepts are not applicable. However, if your visualizations or modules are to be run on both the IBM Visualization Data Explorer and IBM Visualization Data Explorer SMP, then these concepts are important for achieving higher performance.

Every module that performs any significant amount of processing is "parallelized"; that is, the module makes use of all processors made available to Data Explorer to operate on the data.

Data Explorer uses explicit data partitioning as the primary framework for parallelism. Data Explorer partitions the data into local, self-contained regions. In general, visualization modules then generate subtasks corresponding to partitions. For more information about partitioning, see Partition in IBM Visualization Data Explorer User's Reference.

In general, parallel programming is complex. To help manage it, Data Explorer simplifies the process by providing a simple fork-join parallelism model to implement coarse-grain shared memory parallelization (data parallel). Using data partitions, read-only objects, and a single-fork join mode simplifies the module writing task by avoiding the explicit use of locks in modules, thereby reducing the possibility of deadlock. For information about adding modules to the Data Explorer system, see IBM Visualization Data Explorer Programmer's Reference.