Missing Values in Bounds/Domain Check filters

JEDI
1

Hi, All,

I’m testing the domain check function using the observable_domain_mask generated according to the instructions in this pull request. (https://github.com/JCSDA-internal/ufo/pull/3343)
The observable_domain_mask variable is generated as expected, with a value of 0 within the model domain and missing values outside the domain. However, when I apply the bounds check filter as follows, JEDI still returns errors for points outside the model domain, despite the observable_domain_mask values are still missing at those points.

 - filter: Bounds Check
    filter variables:
    - name: windEastward
    - name: windNorthward
    test variables:
    - name: GeoVaLs/observable_domain_mask
    flag all filter variables if any test variable is out of bounds: true
    treat missing as out of bounds: true
    minvalue: 0
    maxvalue: 0.01

When I switched to Domain Check filter as shown below:

  - filter: Domain Check
    where:
    - variable:
        name: GeoVaLs/observable_domain_mask
      minvalue: 0.0
      maxvalue: 0.1

It worked fine until I added additional post-filter QC options. The issues in those out-of-domain data points with missing observable_domain_mask values are back.

Could anyone provide insights into what might be causing issues with identifying missing values in the Check filters and what could be causing conflicts with these filters?

Many Thanks,
Xu

2

Thanks Xu for posting this issue.
For some reasons, the “Bounds check” filter has a problem in recognizing the missing value of the “observable_domain_mask” for some OBS locations (e.g., the 1st OBS in the OBS sequence) far away from the HAFS domain, but the “domain check” filter can recognize the missing value and has NO problem. The “Bounds check” filter also works fine in the testing cases with MPAS-JEDI, see https://github.com/JCSDA-internal/ufo/pull/3343. I am wondering if the “missing value” of JEDI might have various values during/after different filterings are applied? The two filters seem work slight differently with regarding to the missing value for these small number of OBS locations. Any insight into how the “missing values” are assigned (potentially different) in JEDI would be very helpful. Could this be a limitation of current JEDI in handling “missing values”? Thanks!