Environmental Fluid Dynamics Code Plus (EFDC+) is a powerful modeling tool widely used for hydrodynamic simulations. However, large and complex models can take significant computational time, sometimes making simulations impractical. To address this, EFDC+ now employs parallel processing capabilities using MPI (Message Passing Interface) combined with OpenMP (Open Multi-Processing), significantly accelerating model simulations. In this tutorial, we discuss how to harness the full potential of parallel processing and showcase its dramatic performance improvements through benchmarks.

Understanding Parallel Processing: MPI and OpenMP

Parallel processing distributes computational workloads across multiple processors or cores, significantly reducing simulation run times. EFDC+ leverages two primary methods:

• MPI (Message Passing Interface): A standardized method used for parallel processing that divides computational tasks across multiple processors, enabling simulations to run on clusters of interconnected computers.
• OpenMP: A simpler parallelization method that uses multiple threads within a single processor or node to distribute computational tasks.

EFDC+ combines these two approaches into a hybrid MPI/OpenMP system, effectively harnessing both methods’ strengths.

EFDC+ Domain Decomposition Approach

EFDC+ utilizes a domain decomposition approach, dividing the simulation domain into multiple smaller subdomains. Each subdomain is processed in parallel, with MPI handling the communication between these subdomains. This approach ensures that computations remain efficient by balancing the workload evenly among processors, minimizing communication overhead, and thus maximizing computational speed.

Key aspects of domain decomposition include:

• Load Balancing: Distributing computational tasks evenly among subdomains.
• Ghost Cells: Cells at subdomain boundaries that exchange data to ensure continuity across subdomains.
• Adaptive Time-Stepping: Synchronizing time steps across subdomains by choosing the minimum timestep among all subdomains.

Benchmarking with the Chesapeake Bay Model

To demonstrate the performance improvements offered by parallel processing, EFDC+ conducted benchmarks using a detailed hydrodynamic model of Chesapeake Bay:

• Model Setup: Nearly 800,000 computational cells, simulating temperature, salinity, and hydrodynamics.
• MPI/OpenMP Hybrid Performance: Achieved a nearly 17-fold speedup with 32 cores compared to the previous OpenMP-only approach, which could only achieve about a 2.5-fold speedup.
• Strong Scaling Study: Conducted using Amazon Web Services (AWS) cluster up to 96 cores, resulting in a maximum speedup of approximately 25 times compared to single-core execution.

These benchmarks clearly illustrate the dramatic performance gains possible with EFDC+’s parallel processing capabilities.

Best Practices for Achieving Optimal Performance

To fully benefit from parallel processing, EFDC+ users should consider the following guidelines:

• Optimal Subdomain Configuration: Utilize EFDC Explorer’s automated tools to balance computational loads effectively across subdomains.
• Computational Environment Setup: Leverage high-speed network interfaces and powerful computing resources, such as those available on cloud platforms like AWS.
• Monitoring and Adjustments: Regularly analyze runtime logs to fine-tune subdomain and thread settings for maximum efficiency.