SeaScope FVON

This page reads summary and detail layers built from NetCDF/CSV exports of our profile pipeline (backend/datamesh.py). The steps below are what we apply when turning raw DataMesh tables into the observation files used here.

1. Quality control (QC = 1)

Whenever the source table provides quality information, we keep a row only if every applicable flag equals 1 (good). That includes:

• any column whose name ends with _quality_control;
• spike checks when present (qc_temp_spike or TEMP_QC);
• position checks when present (qc_location or QC_LOCATION).

If none of these columns exist, we do not apply this QC mask at this stage.

2. Upcast / ascent leg

We aim to use the part of the cast where the sensor is coming back up through the water column, which is usually more stable for temperature.

• North Atlantic Arc (NAA) FV NRT: when segment_type (or SEGMENT_TYPE) is present, we keep only segment code 2 (upcast). Codes 1 and 3 are downcast and soak/hold respectively.
• Other programmes: if we did not already filter by segment type, we trim each profile after its maximum depth: we detect a sustained decreasing depth trend (rolling mean over a short window) and keep samples from that ascent onward, in line with the Moana / fishSOOP-style upcast logic.

3. Clustering by location (per instrument)

For each instrument or platform identifier, we run DBSCAN on positions mapped to the unit sphere so distances respect the Earth’s curvature (not raw degrees in flat lon/lat). Nearby fixes within about 0.01° great-circle separation group into the same cluster. Each cluster becomes part of the profile identifier (…_cluster_<label>), so a vessel reporting from slightly different berths or drift between hauls does not collapse into one ambiguous profile unless it is within that tolerance.

For NAA, we optionally combine vessel and serial identifiers so clustering keys stay consistent across columns.

4. Standard fields and cleaning

• We map time, latitude, longitude, depth or pressure, and temperature from the source column names we recognise.
• If only pressure is available, we convert to depth in metres with a latitude-dependent UNESCO-style formula, then take the absolute value as depth below the surface.
• Longitude is normalised to a 0–360° range for consistency.
• If the source has no explicit profile id, we synthesise one from the observation time (UTC, floored to 10 minutes) and latitude/longitude rounded to two decimals.
• Rows with missing time, position, depth, or temperature are dropped.

5. Model reference temperature (GLORYS / Copernicus)

We download daily Copernicus global fields (thetao) for the pipeline cycle and match each observation to the model in three steps:

• Nearest wet column: we build a spatial index of grid points where the model has valid (non-NaN) ocean temperature after reducing extra dimensions. Your observation is assigned to the nearest such wet neighbour in 3D space on the sphere, not to the nearest land mask corner.
• Nearest model time: we take the closest daily model time to the observation timestamp.
• Vertical interpolation: along that column we linearly interpolate thetao to the observation depth (handling sign conventions for the model’s vertical coordinate). The result is stored as glorys_temp_c in the observation files when the lookup succeeds; otherwise the field stays empty for that point.

6. Forecast files and map summaries

Model grids for the cycle are merged from per-day NetCDF chunks with coordinates sorted and longitudes wrapped consistently. Summary layers you see on the map are derived from these harmonised observations and programme metadata (for example default programme labels where the source does not provide one).

Processing is intended for operational visualisation and download; always refer to provider documentation and flags in the original ERDDAP or DataMesh source for authoritative QC definitions.