Models for EFDC+ Testing
In a previous blog, we outlined how our automated integration testing works at a high level. We highlighted the need to test the code using a batch of test cases to ensure code integrity and increase confidence in modeling results and how that is now done using an automated testing process. In this post, we will go over the collection of models used for testing and how they are used.
We currently have around sixty different models grouped into three separate tiers. Tier 1 models are designed to run quickly. At a minimum, these models run whenever a commit is pushed to any of our permanent branches. Tier 2 models are generally more complex, real-world models that may take a few hours to run. These are run on a monthly schedule and often several times in the lead-up to a release, using the latest builds of EFDC+. Tier 3 contains a collection of models that generally require several days of runtime. These are not a part of the test automation and will be excluded from this review.
The goal of these models is to provide solid test coverage for the major features of EFDC+. As stated previously, they need to run quickly and demonstrate the targeted model behavior in the limited run period. Tier 1 is arguably the most important group, as these models will likely quickly uncover any major issues with a code update. The vast majority of these models are available for download from our website as test case or demonstration models. All such models can be run using EEMS in the free demo mode. Below is a table of the complete model list for Tier 1 models, which shows at a glance the various modules being tested and relevant settings. Table 1 is helpful for identifying which modules or features are being tested and detecting gaps in the testing process.
In contrast to most Tier 1 models, the Tier 2 models are more complex and help us provide better test coverage for how various components interact with one another over an extended period. While some of these models are available on our website, most were developed to support clients and are more specialized in the features they test for. The current list of Tier 2 models is provided in the table below.
Our collection of test models is far from complete, and we are constantly working on increasing our test coverage by adding new test cases and real-world models. We are currently working on implementing a group of models for regular validation of MPI functionality. If you have a model that you think would fit into our testing program and want to ensure its feature set is covered in our testing, please contact us at support@eemodelingsystem.com. In an upcoming blog post, we will look at the criteria used for identifying differences in model runs. The blog will also describe how we assess the impact of the change in the code and determine if it is acceptable.
