Contributor internals
These modules are documented to make scientific review and maintenance easier. Names beginning with an underscore are implementation details and may change without a compatibility promise.
Ingestion
Retrieve and align observed MetObs and modelled STRÅNG weather data.
SMHI exposes station observations and gridded solar radiation through separate
APIs with different payloads and missing-value conventions. This module hides
those transport details: it discovers full-year stations, filters observation
quality flags, converts -999 STRÅNG sentinels to missing values, and aligns
all sources on one continuous hourly UTC grid. Raw responses can be cached
atomically to make repeated educational and production runs kind to the APIs.
- class smhi2epw.ingestion.StationMeta(station_id, name, latitude, longitude, elevation=0.0, wmo_id='999999')[source]
Bases:
objectDescribe the MetObs station supplying observed weather.
- Parameters:
- station_id
Positive SMHI station identifier.
- Type:
- name
Human-readable station name with API instruction text removed.
- Type:
- latitude, longitude
Station coordinates in decimal degrees.
- elevation
Station height in metres above sea level.
- Type:
- wmo_id
Genuine WMO identifier, or
"999999"when SMHI supplies none.- Type:
- distance_km(lat, lon)[source]
Calculate great-circle distance to a target coordinate.
- Parameters:
- Returns:
Haversine distance in kilometres, using mean Earth radius 6371.0088 km.
- Return type:
Examples
>>> station = StationMeta(71420, "Göteborg A", 57.7156, 11.9924) >>> station.distance_km(57.7156, 11.9924) 0.0
- class smhi2epw.ingestion.CachedClient(cache_dir=None, timeout=60.0, session=None, max_retries=4, cache_ttl=None, refresh=False)[source]
Bases:
objectThin
requestswrapper that caches raw payloads on disk.Caching makes recompilation repeatable and avoids redundant heavy queries against SMHI. Transient HTTP failures are retried with exponential backoff, cached entries can optionally expire, and writes use temporary sibling files followed by atomic replacement.
- Parameters:
cache_dir (Optional[str]) – Cache directory or
Noneto disable disk caching.timeout (float) – Per-request timeout in seconds.
session (Optional[requests.Session]) – Optional preconfigured
requests.Session, useful for tests.max_retries (int) – Retry count for GET responses with status 429, 500, 502, 503, or 504.
cache_ttl (Optional[float]) – Maximum cache age in seconds, or
Nonefor no expiry.refresh (bool) – Ignore cached responses when
True.
Examples
>>> client = CachedClient(cache_dir=".smhi_cache", cache_ttl=86400) >>> isinstance(client.session, requests.Session) True
- scoped()[source]
Share transport/cache while isolating receipts for one compilation.
Each clone owns its receipt dictionary and lock. Worker threads within a compilation share that clone; concurrent compilations cannot clear or contaminate one another’s provenance.
- Return type:
CachedClient
- static _build_session(max_retries)[source]
Construct a GET-only retrying HTTP session.
- Parameters:
max_retries (int) – Total retry budget. Values at or below zero disable adapters.
- Returns:
Session with identical retry adapters for HTTP and HTTPS.
- Return type:
- _cache_path(url, suffix)[source]
Map a URL to a stable, filesystem-safe cache path.
The first 24 hexadecimal SHA-256 characters avoid exposing query strings in filenames while making the same URL deterministic.
Noneis returned when caching is disabled.
- _cache_valid(path)[source]
Return whether a cache file exists and satisfies refresh/TTL policy.
- get_text(url, suffix='txt', *, log_context=None)[source]
Return a text response from cache or HTTP.
- Parameters:
- Returns:
UTF-8 cached text or
requests.Response.text.- Return type:
- Raises:
IngestionError – If all HTTP attempts fail.
Notes
Cache writes are atomic. Readers therefore observe either the previous complete payload or the new complete payload, never a partial write.
- _record_response(url, suffix, text, *, cached)[source]
Pin the exact parser input for weather calibration provenance.
- smhi2epw.ingestion._select_position(positions, year)[source]
Pick the station position record valid during
year.- Parameters:
- Returns:
Matching position, or the most recent record as a documented fallback.
- Return type:
- Raises:
IngestionError – If the station exposes no position records.
- smhi2epw.ingestion._clean_station_name(title, station_id)[source]
Extract a tidy station name from a metobs parameter-station title.
Titles look like
"Lufttemperatur - Göteborg A: Välj tidsutsnitt"; the station name is the segment between the parameter prefix (” - “) and the trailing “: …” instruction.Examples
>>> _clean_station_name("Lufttemperatur - Göteborg A: Välj", 71420) 'Göteborg A' >>> _clean_station_name(None, 71420) 'Station 71420'
- smhi2epw.ingestion.get_station_metadata(station_id, client, year=None, *, parameter=1, log_context=None)[source]
Resolve a station’s name, coordinates and elevation from metobs.
- Parameters:
station_id (int) – SMHI MetObs station identifier.
client (CachedClient) – Cache-aware transport used for the metadata request.
year (int | None) – Optional historical year used to select the correct position record.
parameter (int) – MetObs parameter whose station metadata is requested; defaults to temperature (1). Donors use their actual borrowed parameter.
- Returns:
Cleaned station identity and year-aware coordinates/elevation.
- Return type:
- Raises:
IngestionError – If the payload shape or position metadata is unusable.
Notes
The SMHI
keyfield is not assumed to be a WMO number. Missing genuine WMO metadata is represented by the EPW fallback"999999".
- smhi2epw.ingestion._haversine_km(lat1, lon1, lat2, lon2)[source]
Return great-circle distance in kilometres between two coordinates.
Examples
>>> round(_haversine_km(57.7, 12.0, 59.3, 18.1)) 396
- smhi2epw.ingestion._stations_for_parameter(param, client)[source]
List stations exposing one MetObs parameter.
Malformed individual entries are ignored so one bad metadata record does not make every station undiscoverable. A non-dictionary response yields an empty mapping for the caller to handle.
- smhi2epw.ingestion._year_bounds_ms(year)[source]
Return inclusive UTC millisecond bounds for a calendar year.
Examples
>>> start, end = _year_bounds_ms(2020) >>> end > start True
- smhi2epw.ingestion._covers_year(record, year)[source]
Return whether a station metadata record covers every instant of a year.
- smhi2epw.ingestion.find_nearest_station(lat, lon, year, client, *, allow_partial=False, log_context=None)[source]
Find the nearest metobs station carrying all required parameters in
year.- Parameters:
lat (float) – Requested point in decimal degrees.
lon (float) – Requested point in decimal degrees.
year (int) – Calendar year that every required parameter must fully cover.
client (CachedClient) – Cache-aware metadata transport.
allow_partial (bool) – Prefer full coverage but accept a station overlapping the year when automatic recovery can supply its remaining variables/hours.
- Returns:
Closest qualifying station with year-aware metadata.
- Return type:
- Raises:
IngestionError – If station lists are unavailable or no station covers all required temperature, humidity, pressure, speed, and direction records.
Notes
No arbitrary candidate-count cutoff is applied: all stations are ranked and examined until a full-year match is found.
- smhi2epw.ingestion.find_nearest_radiation_station(lat, lon, year, client)[source]
Return
(station_id, distance_km)for the nearest Sol (pyranometer) station.- Parameters:
- Returns:
Station ID and distance in kilometres, or
Nonewhen automatic measured radiation is unavailable. Distance policy is applied later bysmhi2epw.compiler.compile_epw().- Return type:
tuple of int and float or None
- smhi2epw.ingestion.radiation_station_distance(station_id, lat, lon, year, client)[source]
Return distance to a full-year explicit radiation station.
Nonemeans the station is absent from parameter 11 metadata or fails to cover the full requested year; the compiler treats that as a hard error for an explicitly requested station.
- smhi2epw.ingestion._parse_metobs_csv(text, column, window, accepted_quality=None)[source]
Parse a corrected-archive CSV into a UTC-indexed series.
- Parameters:
text (str) – Corrected-archive semicolon-delimited response, including preamble.
column (str) – Canonical output series name.
window (Tuple[Timestamp, Timestamp]) – Inclusive UTC start/end timestamps retained from the response.
accepted_quality (set | None) – Accepted SMHI quality codes; defaults to
{"G", "Y"}.
- Returns:
Sorted, duplicate-free, timezone-aware observations. Rejected quality values remain
NaNso processing diagnostics see the gap.- Return type:
- Raises:
IngestionError – If no data header or value column can be identified.
Notes
Decimal commas are accepted. Duplicate timestamps keep the final published value, matching corrected-archive semantics.
- smhi2epw.ingestion.fetch_metobs_parameter(station_id, param, column, window, client, accepted_quality=None, *, log_context=None)[source]
Fetch one station parameter from the corrected MetObs archive.
Parameters mirror
_parse_metobs_csv();paramandstation_idform the endpoint URL. The returned series is hourly UTC where supplied by SMHI, but final resampling and grid alignment occur iningest(). Parameter 16cloud_coveris normalized to octas using the CSV preamble’s explicitEnhetdeclaration. Unknown/missing cloud units raiseIngestionError; ingestion treats that optional source as unavailable.
- smhi2epw.ingestion._normalize_cloud_cover(text, series)[source]
Convert declared MetObs cloud units to octas without magnitude guessing.
Enhetmust belong to the parameter metadata header before the hourly data block. Percent/procent values use their source 0–100 bounds before conversion; octa/okta values retain source 0–8 bounds. Invalid observations become gaps. Original quality filtering and corrected-archive duplicates remain those of_parse_metobs_csv().
- smhi2epw.ingestion.fetch_strang_parameter(lat, lon, param, column, window, client, *, log_context=None)[source]
Fetch one STRÅNG irradiance parameter for a point and UTC window.
- Parameters:
lat (float) – Requested solar coordinates in decimal degrees.
lon (float) – Requested solar coordinates in decimal degrees.
param (int) – STRÅNG parameter ID, such as 117 for GHI.
column (str) – Canonical series name.
window (Tuple[Timestamp, Timestamp]) – Inclusive UTC start and end timestamps.
client (CachedClient) – Cache-aware JSON transport.
- Returns:
Sorted UTC irradiance values in W/m². STRÅNG’s
-999sentinel is converted toNaN.- Return type:
- Raises:
IngestionError – If the endpoint does not return the expected list payload.
- smhi2epw.ingestion.ingest(meta, year, client, max_workers=8, utc_offset=0.0, accepted_quality=None, radiation_station_id=None, solar_latitude=None, solar_longitude=None, radiation_required=False, tolerate_failures=False, *, log_context=None)[source]
Query both endpoints in parallel and return a UTC-indexed DataFrame.
- Parameters:
meta (StationMeta) – Station supplying required meteorological observations.
year (int) – Actual meteorological year.
client (CachedClient) – Cache-aware SMHI transport.
max_workers (int) – Maximum concurrent parameter requests.
utc_offset (float) – Whole-hour LST offset used only to size the year-boundary buffer.
accepted_quality (set | None) – Optional override for usable MetObs quality flags.
radiation_station_id (int | None) – Optional Sol station supplying measured GHI.
solar_latitude (float | None) – Requested STRÅNG point; station coordinates are fallbacks when absent.
solar_longitude (float | None) – Requested STRÅNG point; station coordinates are fallbacks when absent.
radiation_required (bool) – Make measured-GHI failure fatal for an explicit radiation station.
tolerate_failures (bool) – Retain unavailable sources as gaps and record failures in frame attrs for automatic recovery. The default retains strict ingestion errors.
- Returns:
Continuous hourly UTC frame spanning the year plus boundary buffer.
- Return type:
- Raises:
IngestionError – If a required source request or payload fails.
Notes
Requests run concurrently but columns are assembled in deterministic canonical order. Optional cloud and automatically selected GHI failures are represented as missing columns. STRÅNG point values are instantaneous at the full hour, so adjacent samples are averaged to represent the preceding EPW interval.
GHI and DNI are requested throughout supported history. Before horizontal beam data begins in April 2017, instantaneous DNI is projected before averaging.
Processing
Fill source gaps and derive the physical quantities required by EPW.
Ingestion intentionally preserves missing observations. This module decides
which gaps can be reconstructed transparently, converts source units, derives
dew point and longwave radiation, and reconciles four possible solar-data
paths. All transformations operate on a continuous hourly UTC frame; Local
Standard Time conversion belongs to smhi2epw.export.
- class smhi2epw.processing.ProcessingReport(interpolated_fraction=<factory>, total_interpolated_fraction=0.0, clamped_dni_hours=0, energy_balance_max_residual=0.0, missing_columns=<factory>, cloud_available=False, solar_source='strang', linear_filled_hours=<factory>, diurnal_filled_hours=<factory>, max_gap_hours=<factory>, bounded_filled_hours=<factory>, invalid_observation_hours=<factory>, primary_missing_hours=<factory>, cross_station_filled_hours=<factory>, gap_fallback_sources=<factory>, gap_fallback_attempts=<factory>, required_reconstructed_fraction=0.0, warnings=<factory>, reanalysis_filled_hours=<factory>, reanalysis_metadata=<factory>, source_fractions=<factory>, weather_classification='observation_based')[source]
Record data filling and solar-quality diagnostics.
- Parameters:
- interpolated_fraction
Original missing fraction for each processed source column.
- total_interpolated_fraction
Mean of the per-column missing fractions.
- Type:
- clamped_dni_hours
Hours whose positive DNI was reduced by horizon or extraterrestrial limits. Routine nighttime zeros are not counted.
- Type:
- energy_balance_max_residual
Maximum absolute interval residual of
GHI - (DHI + dirh)in W/m². Mean DNI times a midpoint cosine need not equal mean horizontal beam.- Type:
- missing_columns
Optional source columns that were absent or entirely missing.
- Type:
List[str]
- cloud_available
Whether usable observed or reconstructed total cloud cover was available.
- Type:
- solar_source
One of
"strang","measured+strang_partition","strang_ghi+erbs","measured+erbs","era5", or"mixed"for solar assembled from different hourly sources.- Type:
- linear_filled_hours, diurnal_filled_hours
Filled-hour counts by column and method.
- bounded_filled_hours
Reconstructed hours clipped to each variable’s representable range.
- primary_missing_hours, invalid_observation_hours
Required-variable missing and invalid counts before source recovery.
- cross_station_filled_hours
Required-variable counts filled from same-hour donor observations.
- gap_fallback_sources, gap_fallback_attempts
Donor receipts and attempted requests, including parameter errors.
- required_reconstructed_fraction
Pre-recovery missing/invalid fraction across required ingestion cells. Populated by automatic or strict donor recovery.
- Type:
- warnings
Structured records containing
code,messageand optionalvariable/details; plausibility warnings retain valid extremes.- Type:
List[dict]
- reanalysis_filled_hours
Weather-variable sample counts supplied by ERA5 on the buffered grid, including optional cloud cover during actual weather replacement hours.
- reanalysis_metadata
Provider, model, grid/elevation, attribution and original overlap comparisons; pressure differences are expressed in Pa.
- Type:
- source_fractions
Source proportions by variable over exported LST hours.
- weather_classification
"observation_based","mixed_reconstructed"or"reanalysis_only". Includes reconstructed GHI; routine physical nighttime zeros do not downgrade otherwise observation-based inputs.- Type:
- smhi2epw.processing._max_gap_length(mask)[source]
Return the longest contiguous run of true values.
Examples
>>> _max_gap_length(np.array([False, True, True, False, True])) 2
- smhi2epw.processing._gap_runs(mask)[source]
Return half-open index pairs for all contiguous true runs.
Examples
>>> _gap_runs(np.array([True, True, False, True])) [(0, 2), (3, 4)]
- smhi2epw.processing._linear_estimate(values, start, end)[source]
Estimate a half-open gap from its immediate endpoint observations.
Two endpoints produce a linear bridge; a single boundary endpoint produces a constant one-sided estimate.
Noneis returned when neither endpoint exists, allowing the caller to raise or leave an optional gap missing.Examples
>>> values = np.array([0.0, np.nan, np.nan, 3.0]) >>> _linear_estimate(values, 1, 3) array([1., 2.])
- smhi2epw.processing._profile_estimate(values, start, end, direction)[source]
Estimate a gap from the preceding or following valid 24-hour profile.
- Parameters:
- Returns:
Endpoint-adjusted repeated profile, or
Nonewhen a complete reference day or gap endpoint is unavailable.- Return type:
numpy.ndarray or None
Notes
The daily profile is repeated through the gap and offset linearly so it meets the observations surrounding the gap, following Lundström (2012) section 4.2.2.
directionis -1 for the preceding profile and +1 for the following profile.
- smhi2epw.processing._fill_scalar_series(series, *, required, solar, short_gap_hours, max_gap_hours)[source]
Fill one scalar series and calculate method-specific diagnostics.
- Parameters:
series (Series) – Hourly scalar values.
required (bool) – Raise when a gap cannot be reconstructed; otherwise retain
NaN.solar (bool) – Use daily profiles for every gap and clamp results nonnegative.
short_gap_hours (int) – Maximum size assigned to linear interpolation for non-solar data.
max_gap_hours (int) – Absolute fill ceiling.
- Returns:
Filled series, linearly filled hours, diurnally filled hours, and longest original gap.
- Return type:
- Raises:
DataGapError – If a required gap exceeds the ceiling or lacks a valid estimate.
Notes
For longer gaps, valid previous and next profiles are blended 50/50. A single valid side is accepted, which is important near year boundaries.
- smhi2epw.processing._fill_wind_direction(series, *, required, short_gap_hours, max_gap_hours)[source]
Fill angular wind direction through unit-vector components.
Direct interpolation would treat 359° and 1° as roughly 180°. Converting directions to sine/cosine components preserves circular continuity; the filled vector is converted back to degrees in
[0, 360).- Returns:
Filled direction and the same diagnostics as
_fill_scalar_series().- Return type:
- Parameters:
Examples
>>> index = pd.date_range("2023-01-01", periods=3, freq="h") >>> values = pd.Series([359.0, np.nan, 1.0], index=index) >>> filled, *_ = _fill_wind_direction( ... values, required=True, short_gap_hours=3, max_gap_hours=48 ... ) >>> round(float(filled.iloc[1])) in {0, 360} True
- smhi2epw.processing.impute(frame, required, optional=None, short_gap_hours=3, max_gap_hours=48, report=None)[source]
Fill meteorological gaps and update diagnostics in-place.
- Parameters:
frame (DataFrame) – Continuous hourly data frame modified in-place.
required (List[str]) – Columns whose unfillable gaps abort processing.
optional (List[str] | None) – Columns that may remain partially or entirely missing.
short_gap_hours (int) – Linear interpolation ceiling, three hours by default.
max_gap_hours (int) – Daily-profile fill ceiling, 48 hours by default.
report (ProcessingReport | None) – Existing report to extend, or
Noneto create one.
- Returns:
Per-column missing fractions, methods, and maximum gaps.
- Return type:
- Raises:
DataGapError – If a required column is absent or a required gap is unfillable.
Notes
Required 1–3 hour gaps are linear. Required 4–48 hour gaps use the endpoint-adjusted daily-profile method. Optional gaps beyond the same ceiling remain missing for honest EPW sentinel output.
- smhi2epw.processing.impute_solar(frame, columns, report=None)[source]
Fill solar gaps without bridging night and day linearly.
Solar gaps up to 48 hours always use same-hour daily profiles, even when only one hour long. Values are clipped nonnegative after filling. Entirely missing required solar columns raise
DataGapError.- Parameters:
frame (DataFrame)
report (ProcessingReport | None)
- Return type:
- smhi2epw.processing.convert_units(frame)[source]
Convert source units to EPW units in-place.
MetObs sea-level pressure is converted from hPa to Pa here. The full pipeline subsequently derives pressure at the target elevation. Missing columns are ignored so the helper remains safe for focused examples.
Examples
>>> frame = pd.DataFrame({"pressure": [1013.25]}) >>> convert_units(frame) >>> float(frame.loc[0, "pressure"]) 101325.0
- Parameters:
frame (DataFrame)
- Return type:
None
- smhi2epw.processing.pressure_at_elevation(sea_level_pa, dry_bulb_c, latitude_deg, elevation_m)[source]
Estimate target surface pressure from SMHI parameter 9 (sea-level QFF).
Invert SMHI’s published reduction: QFF = P * exp(H * B / T1). https://www.smhi.se/kunskapsbanken/meteorologi/lufttryck/hur-mats-lufttryck T1 is SMHI’s piecewise effective layer temperature, not virtual temperature. The target height approximates barometer height; station-observed temperature represents the target. This is a derived estimate, not a pressure observation. Inputs have already passed filling and unit conversion. No clipping is used.
- smhi2epw.processing.dew_point(dry_bulb, relative_humidity)[source]
Calculate dew-point temperature with the Magnus approximation.
- Parameters:
- Returns:
Dew-point temperature in degrees Celsius, aligned to the inputs.
- Return type:
Notes
The coefficients
b=17.625andc=243.04 °Cprovide a common Magnus approximation over normal near-surface weather conditions.Examples
>>> round(float(dew_point(pd.Series([20.0]), pd.Series([50.0])).iloc[0]), 1) 9.3
- smhi2epw.processing.horizontal_ir(dry_bulb_c, dew_point_c, cloud_cover_tenths=None)[source]
EnergyPlus-compatible down-welling longwave sky radiation (W/m²).
- Parameters:
- Returns:
Horizontal infrared radiation intensity in W/m².
- Return type:
Notes
EnergyPlus estimates clear-sky emissivity as
0.787 + 0.764*ln(Tdew/273)and applies a cubic cloud multiplier. SMHI parameter 16 is total rather than opaque cloud cover. When supplied, it is used only as an explicit proxy in the cloud-amplification term; the EPW opaque-cover field remains missing.Examples
>>> value = horizontal_ir(pd.Series([20.0]), pd.Series([10.0])).iloc[0] >>> round(float(value), 1) 341.2
- smhi2epw.processing.sky_cover_tenths(cloud_octas)[source]
Convert SMHI total cloud cover (octas 0-8) to EPW tenths (0-10).
Values are rounded to the nearest tenth and clamped to [0, 10]. NaNs are preserved so the exporter can emit the EPW missing token.
Examples
>>> sky_cover_tenths(pd.Series([0.0, 4.0, 8.0])).tolist() [0.0, 5.0, 10.0]
- smhi2epw.processing._finalize_solar(frame, ghi, measured, lat, lon)[source]
Finalize one solar group without replacing valid interval means.
Supplied DNI and beam-horizontal are separate hourly averages: their product identity applies instantaneously, not at one hourly midpoint. Only inferred DNI uses the low-angle inversion guard. Missing horizontal beam with supplied DNI uses mean positive interval geometry as an estimate.
- smhi2epw.processing.apply_solar(frame, lat, lon, report=None)[source]
Finalize solar components using their preceding-hour interval semantics.
Supplied DNI is retained at low solar elevation. Supplied horizontal beam closes GHI with DHI; measured GHI scales the model component group. Erbs decomposition and a five-degree inversion guard apply only without usable supplied DNI. Extraterrestrial caps and wholly dark intervals remain physical constraints. Automatic recovery already uses this same finalizer.
- Parameters:
frame (DataFrame) – Hourly UTC frame modified in place. It must contain GHI and may contain measured GHI, supplied DNI, and direct-horizontal irradiance.
lat (float) – Requested solar coordinates in decimal degrees.
lon (float) – Requested solar coordinates in decimal degrees.
report (ProcessingReport | None) – Optional diagnostics object updated with source, clamping, and closure.
- Return type:
None
Notes
DHI closes against the supplied or estimated horizontal beam interval mean. Mean DNI times a midpoint cosine need not equal mean horizontal beam when DNI or solar geometry changes within the hour. Measured GHI transfers the model partition by scaling both DNI and horizontal beam by the GHI ratio.
- Raises:
KeyError – If no GHI column exists; callers normally prevent this through
impute_solar().- Parameters:
frame (DataFrame)
lat (float)
lon (float)
report (ProcessingReport | None)
- Return type:
None
- smhi2epw.processing.process(frame, lat, lon, *, measured_required=False, target_elevation_m=0.0, report=None)[source]
Run the full processing pipeline in-place and return diagnostics.
- Parameters:
frame (DataFrame) – Continuous hourly UTC ingestion frame, modified in-place.
lat (float) – Requested solar point in decimal degrees.
lon (float) – Requested solar point in decimal degrees.
target_elevation_m (float) – Height used to estimate surface pressure from sea-level QFF. Low-level callers default to sea level; compile_epw always supplies the site height.
measured_required (bool) – If
True, unfillable measured GHI raises instead of falling back to STRÅNG.report (ProcessingReport | None) – Optional diagnostics from nearby-observation recovery, updated in place.
- Returns:
Complete gap, cloud, source, clamp, and closure diagnostics.
- Return type:
- Raises:
DataGapError – If required meteorological/solar data cannot satisfy the fill policy.
Notes
Adds
dew_point,horizontal_ir,dni,dhi,etrh/etrnand (when cloud data is present)sky_covercolumns, and converts units. The inputframemust be indexed by a continuous hourly UTC grid.Measured GHI from an automatically selected station is opportunistic: an unfillable series is discarded with a warning and STRÅNG remains usable. Explicit measured GHI is a user requirement and therefore fails clearly.
Observation recovery and provenance
Assess nearby same-year observations before reconstructing primary gaps.
- smhi2epw.gap_recovery.valid_observations(series, column)[source]
Retain finite source observations within representable physical bounds.
- smhi2epw.gap_recovery.hourly_observations(series, column)[source]
Aggregate valid observations hourly, preserving angular wind continuity.
- smhi2epw.gap_recovery.unresolved(series, column)[source]
Locate missing hours after giving short interpolation first priority.
Assessed donors precede daily profiles for gaps longer than three hours. This preview never modifies the original observations used for assessment. Cancelling short wind vectors remain eligible for donor recovery.
- smhi2epw.gap_recovery._differences(predicted, primary, column)[source]
Calculate signed scalar differences or shortest angular differences.
- smhi2epw.gap_recovery._intervals(mask)[source]
Describe contiguous selected hours using inclusive UTC timestamps.
- smhi2epw.gap_recovery._assessment(primary, donor, column, start, end)[source]
Validate raw transfer and scalar offsets against original observations.
Each gap uses paired observations within thirty days of its boundaries. Three complete UTC validation days represent early, middle, and late overlap. Their adjacent days are excluded from each training fold to avoid fitting on observations immediately neighbouring the held-out day. Only scalar median offsets that improve held-out MAE by at least ten percent are eligible; angular wind direction always uses raw transfer.
- smhi2epw.gap_recovery._warn(report, code, message, column, **details)[source]
Append a structured warning describing donor recovery decisions.
- smhi2epw.gap_recovery.recover(frame, station, year, latitude, longitude, client, *, max_distance_km=75.0, max_stations=3, report=None)[source]
Fill unresolved primary gaps only with assessed nearby observations.
Valid original primary observations remain untouched and provide all training and validation data. Each parameter resolves its own donor metadata, permitting partial stations without temperature observations. Pressure stays in sea-level QFF hPa for subsequent target-site conversion. Short interpolation retains first priority. Insufficient overlap and unsuitable transfers leave gaps for bounded daily profiles and, in automatic mode, reanalysis. The caller applies bounded temporal filling.
- Parameters:
report (ProcessingReport | None)
Fetch explicitly selected ERA5 hourly weather through Open-Meteo.
The public archive service is noncommercial and rate limited. Responses use UTC instantaneous meteorology and preceding-hour mean solar radiation. This adapter adds neither a new dependency nor implicit model selection.
- smhi2epw.reanalysis.fetch(latitude, longitude, elevation, index, client)[source]
Return unit-checked ERA5 values aligned to the buffered UTC grid.
Request dates include both calendar boundaries required by Local Standard Time export. Surface pressure is returned in Pa at the provider’s resolved elevation; cloud percentage is converted to SMHI-compatible octas. Missing hourly values remain NaN for final source selection and validation.
Select complete same-year weather without concealing reconstruction.
This is a source hierarchy, not a weather generator. Original observations, limited temporal estimates, assessed donors and ERA5 are kept distinguishable. Extreme but representable weather is preserved; physically inconsistent solar and unavailable required data are repaired or reported explicitly.
- smhi2epw.automatic.warn(report, code, message, **details)[source]
Append a structured, serializable warning to the quality report.
- Parameters:
report (ProcessingReport)
code (str)
message (str)
- Return type:
None
- smhi2epw.automatic._temporal(series, column, report)[source]
Fill solar gaps under the temporal ceiling and preserve diagnostics.
- Parameters:
series (Series)
column (str)
report (ProcessingReport)
- Return type:
- smhi2epw.automatic._dark_hours(index, lat, lon)[source]
Identify preceding intervals entirely below the geometric solar horizon.
- Parameters:
index (DatetimeIndex)
lat (float)
lon (float)
- Return type:
- smhi2epw.automatic._solar_inputs(frame, report, lat, lon)[source]
Clean radiation, fill bounded gaps and prefer measured hourly GHI.
- smhi2epw.automatic._model_overlap(originals, model, lat, elevation, report)[source]
Report ERA5 disagreement against original observations without rejecting extremes.
- Parameters:
originals (dict)
model (DataFrame)
lat (float)
elevation (float)
report (ProcessingReport)
- Return type:
None
- smhi2epw.automatic.prepare(frame, station, year, lat, lon, elevation, client, report, *, max_distance_km, max_stations, log_context=None)[source]
Fill unresolved inputs and return resolved elevation and source tags.
Pressure remains QFF until processing; reanalysis surface pressure travels in its own Pa column. Solar replacement uses one coherent component group. Per-hour source tags are internal and reduced over the exported LST grid.
- smhi2epw.automatic._finish_solar(frame, ghi, measured, tags, lat, lon, report)[source]
Close radiation per hour using provided direct values or Erbs estimates.
- smhi2epw.automatic.summarize(tags, frame, report)[source]
Reduce exported-hour source tags and warn about abrupt source changes.
- Parameters:
tags (DataFrame)
frame (DataFrame)
report (ProcessingReport)
- Return type:
None
Machine-readable AMY receipts; see docs/provenance.rst and SMHI archives.
Only payloads requested by this compilation are recorded. A hash covers the UTF-8 response consumed by the parser, not unrelated files in a shared cache.
- smhi2epw.provenance.write_provenance(config, result, client)[source]
Atomically save configuration, used payload fingerprints and EPW quality.
Only clients explicitly providing compilation-scoped recording can claim complete raw-response coverage. Custom clients remain usable, but an unscoped lifetime collection is omitted to avoid unrelated requests.
The UTF-8 JSON sidecar replaces an existing file only after serialization succeeds. Filesystem and serialization errors propagate to the caller, which determines whether to warn or fail under its weather policy. The already-written EPW is never modified by a sidecar failure.
Solar geometry
Calculate solar geometry and split global irradiance into EPW components.
Solar position, extraterrestrial irradiance, and Erbs-Driesse decomposition use pvlib. Timestamps must be timezone-aware; the compiler integrates geometry over each preceding hour. Supplied components retain their own interval means.
- smhi2epw.solar.solar_zenith(index, latitude, longitude)[source]
Calculate solar zenith angle for timezone-aware timestamps.
- Parameters:
index (DatetimeIndex) – Timezone-aware timestamps. They are converted to UTC internally.
latitude (float) – Observer coordinates in decimal degrees; north and east are positive.
longitude (float) – Observer coordinates in decimal degrees; north and east are positive.
- Returns:
Zenith angle in degrees and its cosine, one value per timestamp.
- Return type:
- Raises:
ValueError – If
indexis timezone-naive.
Notes
Zenith is 0° when the sun is directly overhead and exceeds 90° below the geometric horizon. pvlib evaluates the NREL SPA with NumPy and estimates delta-T for each UTC year and month. We use true geometric zenith, without atmospheric refraction, at the default sea-level altitude.
Examples
>>> index = pd.DatetimeIndex(["2023-06-21 12:00"], tz="UTC") >>> zenith, cosine = solar_zenith(index, 57.7156, 11.9924) >>> bool(30 < zenith[0] < 40) True >>> bool(cosine[0] > 0) True
References
Reda, I., & Andreas, A. (2004). Solar position algorithm for solar radiation applications. Solar Energy, 76(5), 577–589.
- smhi2epw.solar.extraterrestrial_radiation(index, cos_zenith)[source]
Calculate extraterrestrial horizontal and direct-normal irradiance.
- Parameters:
index (DatetimeIndex) – Timezone-aware timestamps used for Earth–Sun distance correction.
cos_zenith – Cosine of solar zenith for each timestamp.
- Returns:
tuple of numpy.ndarray – Extraterrestrial horizontal and direct-normal irradiance in W/m².
Direct-normal is the solar constant scaled by the Earth-Sun distance
correction using pvlib’s ASCE method and the existing 1367 W/m² constant;
horizontal is that projected onto the horizontal plane (>= 0).
Both quantities are zero when the sun is below the horizon, matching EPW
convention.
Examples
>>> index = pd.DatetimeIndex(["2023-06-21 12:00"], tz="UTC") >>> horizontal, normal = extraterrestrial_radiation(index, [0.8]) >>> round(float(horizontal[0] / normal[0]), 1) 0.8
- smhi2epw.solar.interval_solar_geometry(index, latitude, longitude)[source]
Estimate geometry and extraterrestrial means over each preceding hour.
Five-minute trapezoidal integration includes partially sunlit intervals. The mean positive cosine is useful when only horizontal radiation is known; supplied hourly DNI and beam-horizontal radiation retain their own means.
- Parameters:
index (DatetimeIndex) – Timezone-aware timestamps labelling the end of each interval.
latitude (float) – Observer coordinates in decimal degrees.
longitude (float) – Observer coordinates in decimal degrees.
- Returns:
Mean positive zenith cosine, mean extraterrestrial horizontal and normal irradiance (W/m²), and a mask of wholly dark intervals.
- Return type:
- smhi2epw.solar.erbs_decomposition(ghi, etrh, cos_zenith)[source]
Estimate DHI and DNI from hourly GHI using pvlib Erbs-Driesse.
- Parameters:
ghi – Nonnegative global horizontal irradiance in W/m².
etrh – Extraterrestrial horizontal irradiance averaged over the same hour.
cos_zenith – Mean positive zenith cosine over the hour. Inputs may broadcast.
- Returns:
tuple of numpy.ndarray – Diffuse horizontal irradiance (DHI) and direct normal irradiance (DNI) in W/m².
Given measured global horizontal irradiance
ghi(W/m²), theextraterrestrial horizontal
etrh(W/m²) and the cosine of the solarzenith angle, estimate the diffuse-horizontal (DHI) and direct-normal
(DNI) components.
Notes
The effective normal input
ETRH/cos_zenithkeeps pvlib’s clearness index equal toGHI/ETRHfor these interval means. Erbs-Driesse smooths the classic Erbs diffuse fraction, changing it by less than 0.0005. DNI follows from horizontal closure. At cosine <= 0.087 or nonpositive ETRH, DNI is zero and all GHI is diffuse. This inversion guard applies to inferred DNI; supplied components are handled separately by processing. The existing helper name and+erbssource labels denote the family.Examples
>>> dhi, dni = erbs_decomposition([500.0], [800.0], [0.7]) >>> round(float(dhi[0] + dni[0] * 0.7), 6) 500.0
References
Erbs, D. G., Klein, S. A., & Duffie, J. A. (1982). Estimation of the diffuse radiation fraction for hourly, daily and monthly-average global radiation. Solar Energy, 28(4), 293–302. Driesse, A., Jensen, A., & Perez, R. (2024). A continuous form of the Perez diffuse sky model for forward and reverse transposition. Solar Energy, 267, 112093. doi:10.1016/j.solener.2023.112093.
Export
Convert processed hourly data into a strict EnergyPlus Weather file.
EPW stores weather in Local Standard Time using hour-ending labels 1–24. This module maps the UTC processing frame onto that grid without daylight-saving transitions, formats all 35 fields, validates the complete document, and uses an atomic replacement so an interrupted write cannot leave a partial file.
- smhi2epw.export.is_leap_year(year)[source]
Return whether
yearis a Gregorian leap year.- Parameters:
year (int) – Four-digit Gregorian calendar year.
- Returns:
Truefor years containing February 29.- Return type:
Examples
>>> is_leap_year(2020), is_leap_year(2100), is_leap_year(2000) (True, False, True)
- smhi2epw.export.expected_rows(year)[source]
Return the number of hourly EPW records required for
year.Examples
>>> expected_rows(2023) 8760 >>> expected_rows(2020) 8784
- smhi2epw.export.shift_to_lst(frame, year, utc_offset)[source]
Relabel a UTC-indexed frame onto the target-year LST hour grid.
- Parameters:
- Returns:
Copy indexed by timezone-naive LST hour-ending timestamps.
- Return type:
- Raises:
ValidationError – If the input is timezone-naive, the offset is fractional, or any required source hour is missing.
The grid runs from year-01-01 01:00 to year+1-01-01 00:00 so that –
each calendar day is represented by hours 1..24, where hour 24 is midnight –
at the end of the day. DST is intentionally ignored (constant offset). –
Examples
>>> source_index = pd.date_range( ... "2020-12-31", periods=8786, freq="h", tz="UTC" ... ) >>> source = pd.DataFrame({"temperature": 0.0}, index=source_index) >>> shifted = shift_to_lst(source, 2021, 1) >>> (len(shifted), shifted.index[0].hour) (8760, 1)
- smhi2epw.export._fmt_float(value, missing, decimals=1)[source]
Format a floating EPW field or its field-specific missing sentinel.
None,NaN, and pandas missing values all producemissingwith the requested number of decimal places.Examples
>>> _fmt_float(3.141, 99.9) '3.1' >>> _fmt_float(float("nan"), 99.9) '99.9'
- smhi2epw.export._fmt_int(value, missing)[source]
Round and format an integer EPW field or its missing sentinel.
Examples
>>> _fmt_int(70.6, 999) '71' >>> _fmt_int(None, 999) '999'
- smhi2epw.export.build_header(city, region, country, wmo_id, latitude, longitude, time_zone, elevation, year, station_id)[source]
Build the eight mandatory EPW header lines.
- Parameters:
city (str) – Location labels written verbatim to the
LOCATIONline.region (str) – Location labels written verbatim to the
LOCATIONline.country (str) – Location labels written verbatim to the
LOCATIONline.wmo_id (str) – Genuine WMO identifier, or
"999999"when none is available.latitude (float) – Requested solar-point coordinates in decimal degrees.
longitude (float) – Requested solar-point coordinates in decimal degrees.
time_zone (float) – Whole-hour LST offset from UTC.
elevation (float) – Resolved target elevation in metres above sea level.
year (int) – Actual meteorological year; determines January 1’s weekday and whether EnergyPlus should read February 29.
station_id (int | str) – SMHI MetObs identifier recorded in provenance comments.
- Returns:
Ordered header lines without trailing newline characters.
- Return type:
- Raises:
ValidationError – If a text field contains a comma or newline that would corrupt CSV.
Notes
The comments preserve the required STRÅNG funding acknowledgment and make clear that the WMO field and SMHI station ID are different identifiers.
Examples
>>> header = build_header( ... "Gothenburg", "VG", "SWE", "999999", 57.7156, 11.9924, ... 1.0, 3.0, 2023, 71420, ... ) >>> len(header) 8 >>> header[-1].startswith("DATA PERIODS,1,1,Data,Sunday") True
- smhi2epw.export._row_iter(frame)[source]
Yield 35-field EPW records from an LST hour-ending frame.
- Parameters:
frame (DataFrame) – Validated LST-indexed data using the package’s canonical weather column names.
- Yields:
str – One comma-separated EPW record without a trailing newline.
- Return type:
Notes
A timestamp at midnight belongs to hour 24 of the preceding calendar day. Unavailable secondary fields use EnergyPlus sentinels rather than invented measurements.
- smhi2epw.export.write_epw(path, header_lines, frame, year, *, log_context=None)[source]
Validate and atomically write a complete EPW file.
- Parameters:
- Returns:
Number of data rows written.
- Return type:
- Raises:
ValidationError – If headers, frame, formatted rows, or filesystem operations fail.
Notes
Data is first written to a temporary sibling file.
os.replace()only updates the destination after every row has been formatted and validated, preserving an existing valid file if an earlier step fails.
- smhi2epw.export._validate_header(header_lines)[source]
Validate EPW header count, order, and line integrity.
- Parameters:
header_lines (List[str]) – Candidate EPW header without newline terminators.
- Raises:
ValidationError – If there are not exactly eight lines, a required keyword is misplaced, or an embedded line break is present.
- Return type:
None
- smhi2epw.export._validate_frame(frame, year)[source]
Validate a processed frame before any output bytes are replaced.
- Parameters:
- Raises:
ValidationError – If the exact LST grid, required columns, finiteness, physical ranges, or dew-point consistency check fails.
- Return type:
None
Notes
Required radiation must be nonnegative, wind speed is limited to the EPW range, and dew point may exceed dry bulb by at most 0.2 °C to tolerate one-decimal output rounding.
Reader and constants
Read EnergyPlus Weather files into analysis-ready pandas data frames.
The compiler writes EPW files, but many teaching and quality-control workflows start by reading an existing AMY or TMY. This module provides a deliberately small reader that preserves the EPW calendar fields, names all 35 columns, and understands the field-specific missing-value sentinels defined by EnergyPlus.
- smhi2epw.reader.EPW_COLUMNS: Final[tuple[str, ...]] = ('year', 'month', 'day', 'hour', 'minute', 'data_source_and_uncertainty_flags', 'dry_bulb', 'dew_point', 'relative_humidity', 'pressure', 'extraterrestrial_horizontal', 'extraterrestrial_direct_normal', 'horizontal_ir', 'ghi', 'dni', 'dhi', 'global_horizontal_illuminance', 'direct_normal_illuminance', 'diffuse_horizontal_illuminance', 'zenith_luminance', 'wind_direction', 'wind_speed', 'total_sky_cover', 'opaque_sky_cover', 'visibility', 'ceiling_height', 'present_weather_observation', 'present_weather_codes', 'precipitable_water', 'aerosol_optical_depth', 'snow_depth', 'days_since_last_snowfall', 'albedo', 'liquid_precipitation_depth', 'liquid_precipitation_quantity')
Canonical names of the 35 EPW data fields, in file order.
- smhi2epw.reader._parse_location(line)[source]
Parse the first EPW header line into friendly location metadata.
- Parameters:
line (str) – Comma-separated
LOCATIONrecord.- Returns:
Text identifiers plus numeric latitude, longitude, UTC offset, and elevation.
- Return type:
- Raises:
ValidationError – If field count, keyword, or numeric conversion is invalid.
- smhi2epw.reader._validate_read_calendar(frame)[source]
Require possible dates and complete ordered hourly annual coverage.
Each representative date must exist in its recorded year. Comparing the remaining calendar fields against a normalized year permits composite TMY years without permitting duplicated, skipped, or reordered hours. The two commonly used hourly minute conventions, zero and sixty, are accepted when one convention is used consistently throughout the file.
- Parameters:
frame (DataFrame)
- Return type:
None
- smhi2epw.reader._validate_read_frame(frame)[source]
Validate EPW row cardinality, calendar fields, and physical ranges.
Missing sentinels must already be represented by
NaN. Missing optional fields are accepted, but every finite value must fall within its EPW range. Both 8760-row common years and 8784-row leap years are supported, including TMY files whose calendar-year column changes between representative months.- Parameters:
frame (DataFrame)
- Return type:
None
- smhi2epw.reader.read_epw(path, *, validate=True, missing_as_nan=True, log_context=None)[source]
Read an AMY or TMY EPW file into a labelled data frame.
- Parameters:
path (str | PathLike[str]) – Path to an EnergyPlus Weather file.
validate (bool) – If
True, require eight standard headers, 35 fields per row, 8760 or 8784 ordered hourly data rows, possible integer calendar fields, consistent minutes of 0 or 60, and valid physical ranges.missing_as_nan (bool) – If
True, replace each field’s EnergyPlus missing sentinel withnumpy.nan. Set this toFalsewhen inspecting the raw encoding.
- Returns:
The 35 EPW fields under
EPW_COLUMNS. EPW calendar columns are preserved rather than converted to an index because TMY files may use different representative years for different months. The original headers, parsed location metadata, and source path are available asframe.attrs['header'],frame.attrs['location'], andframe.attrs['source_path'].- Return type:
- Raises:
ValidationError – If the file cannot be decoded as a structurally valid EPW file.
Notes
EPW hours are numbered 1 through 24 and describe the interval ending at that hour. Thus hour 1 represents 00:00–01:00 and hour 24 represents 23:00–24:00 in local standard time.
Examples
>>> from smhi2epw import read_epw >>> weather = read_epw("gothenburg_2023.epw") >>> weather[["dry_bulb", "ghi"]].describe() >>> weather.attrs["location"]["city"] 'Gothenburg'
Centralize API semantics, physical constants, and EPW sentinel values.
Keeping these values in one dependency-free module makes the assumptions used by ingestion, processing, and export easy to audit. Parameter mappings translate SMHI numeric IDs to canonical internal column names. Missing-value dictionaries are field-specific because EPW does not use one universal missing token.
STRÅNG GHI and direct-normal radiation are available from 1999. The separately published direct-horizontal field begins on 18 April 2017; earlier hours derive it from instantaneous DNI and solar geometry before interval averaging.
CLI
Expose the high-level compiler as the smhi2epw shell command.
The CLI is intentionally a thin adapter: build_parser() owns argument
spelling and help text, while main() translates parsed values into an
EPWConfig. Scientific work remains in the package
modules, so CLI and Python users receive identical results.
- smhi2epw.cli.build_parser()[source]
Construct the command-line argument parser.
- Returns:
Parser for station selection, target coordinates, caching, radiation selection, and Local Standard Time settings.
- Return type:
Examples
>>> parser = build_parser() >>> args = parser.parse_args(["2023", "out.epw", "--station", "71420"]) >>> (args.year, args.station) (2023, 71420)
- smhi2epw.cli.main(argv=None)[source]
Run the command-line compiler and return a process exit code.
- Parameters:
argv – Optional argument sequence without the program name.
Nonereadssys.argv, matching normal console-script behavior.- Returns:
0after a successful write or1for a handled package error. Invalid argparse syntax raisesSystemExitwith code2.- Return type:
Examples
Programmatic callers can test help without starting a subprocess:
main(["--help"]) # raises SystemExit(0)
A real compilation contacts SMHI and writes the requested path:
main(["2023", "out.epw", "--station", "71420"]) # doctest: +SKIP