The forecast module#

EMHASS will need 4 forecasts to work properly:

  • PV power production forecast (internally based on the weather forecast and the characteristics of your PV plant). This is given in Watts.

  • Load power forecast: how much power your house will demand over the configured forecast horizon. This is given in Watts.

  • Load cost forecast: the price of the energy from the grid over the configured forecast horizon. This is given in currency/kWh.

  • PV production selling price forecast: the price at which you will sell your excess PV production over the configured forecast horizon. This is given in currency/kWh.

Some methods are generalized to the 4 forecasts needed. For all the forecasts it is possible to pass the data either as a passed list of values or by reading from a CSV file. With these methods, it is then possible to use data from external forecast providers.

Then there are the methods that are specific to each type of forecast and where the proposed forecast is treated and generated internally by this EMHASS forecast class.

For the weather forecast, the first method (open-meteo) uses Open-Meteo weather forecast API, which proposes detailed forecasts based on Lat/Lon locations. Another method (solcast) is using the Solcast PV production forecast service. A final method (solar.forecast) is using another external service: Solar.Forecast, for which just the nominal PV peak installed power should be provided. Search the forecast section on the documentation for examples of how to implement these different methods.

The get_power_from_weather method is proposed here to convert irradiance data to electrical power. The PVLib module is used to model the PV plant. A dedicated web app will help you search for your correct PV module and inverter: https://emhass-pvlib-database.streamlit.app/

The specific methods for the load forecast is a first method (typical) that uses historic values of a typical household power consumption. This method uses simple statistic methods and load power grouped by the same day-of-the-week of the current month. The load power is scaled using the parameter maximum_power_from_grid. A second method (naive) that uses a naive approach, also called persistence. It simply assumes that the load at a given time of day will be the same as the last time that time of day was observed, typically 24 hours earlier. A third method (mlforecaster) uses an internal custom forecasting model using machine learning. There is a section in the documentation explaining how to use this method.

Note

This custom machine learning model is introduced from v0.4.0. EMHASS proposed this new mlforecaster class with fit, predict and tune methods. Only the predict method is used here to generate new forecasts, but it is necessary to previously fit a forecaster model and it is a good idea to optimize the model hyperparameters using the tune method. See the dedicated section in the documentation for more help.

For the PV production selling price and Load cost forecasts the privileged method is a direct read from a user-provided list of values. The list should be passed as a runtime parameter during the curl to the EMHASS API.

PV power production forecast#

open-meteo#

The default method for PV power forecast is the weather forecast API proposed by Open-Meteo. For more detail see the Open-Meteo API documentation. This is obtained using method=open-meteo. This site proposes detailed forecasts based on Lat/Lon locations. The weather forecast data is then converted into PV power production using the pv_module_model and pv_inverter_model parameters defined in the configuration.

solcast#

The second method uses the Solcast solar forecast service. Go to https://solcast.com/ and configure your system. You will need to set method=solcast and use two parameters solcast_rooftop_id and solcast_api_key that should be passed as parameters at runtime or provided in the configuration/secrets. solcast_rooftop_id can hold several rooftop IDs, separated by commas or spaces; the first one is plant 0 (see PV plants and the per-plant series). The free hobbyist account will be limited to 10 API requests per day, the granularity will be 30 minutes and the forecast will be updated every 6 hours. If needed, better performances may be obtained with paid plans: https://solcast.com/pricing/live-and-forecast.

For example:

# Set weather_forecast_method parameter to solcast in your configuration (configuration page / config_emhass.yaml)
weather_forecast_method: 'solcast'

# Example of running day-ahead, passing Solcast secrets via runtime parameters (i.e. not set in configuration)
curl -i -H "Content-Type:application/json" -X POST -d '{
	"solcast_rooftop_id":"<your_system_id>",
	"solcast_api_key":"<your_secret_api_key>"
}' http://localhost:5000/action/dayahead-optim

Multi-day Solcast forecasts for longer MPC horizons#

When weather_forecast_method: solcast is set and a prediction_horizon longer than one day is passed to naive-mpc-optim, EMHASS automatically extends the Solcast forecast window to cover the full horizon. The Solcast API request scales its hours= parameter accordingly, so no extra configuration is needed — simply pass the desired prediction_horizon at runtime:

# Run a 36-hour naive-MPC with a native Solcast forecast (no manual delta_forecast_daily required)
curl -i -H "Content-Type:application/json" -X POST -d '{
    "prediction_horizon": 72,
    "solcast_rooftop_id": "<your_system_id>",
    "solcast_api_key": "<your_secret_api_key>"
}' http://localhost:5000/action/naive-mpc-optim

Solcast returns a multi-day forecast (several days at 30-minute resolution), so a 36-hour horizon is comfortably within range; on the free hobbyist tier the practical limit is the daily API-call quota rather than the forecast length. If a prediction_horizon ever runs past the end of the data Solcast returns, those trailing steps are zero-filled (PV set to zero), so longer horizons are still handled safely.

When the horizon auto-extends, any forecast passed as a runtime list (weather_forecast_method: list, load_power_forecast, load_cost_forecast, or prod_price_forecast) must cover the full prediction_horizon steps; a shorter list is rejected with an error rather than being silently truncated to one day.

Conservative PV bias (P10 blend)#

Solcast returns three probabilistic estimates for each forecast period: P50 (central / median), P10 (low / conservative, 10th percentile), and P90 (high / optimistic, 90th percentile). By default EMHASS uses only the P50 estimate. The weather_forecast_pv_quantile_bias parameter lets you blend P50 and P10 so the optimizer plans against a more cautious solar outlook.

This is useful when the cost of a solar shortfall outweighs the cost of over-reserving the battery. Forecast error is asymmetric: if the sun underperforms the central estimate you may have to buy back the shortfall at the peak retail rate, which usually costs more than the value you give up by holding a little extra reserve. Biasing toward P10 makes the plan robust to that downside.

The blend formula applied per period is:

estimate = bias * pv_estimate10 + (1 - bias) * pv_estimate

where bias is weather_forecast_pv_quantile_bias (range 0–1, default 0).

weather_forecast_pv_quantile_bias

Effective estimate

Effect on plan

0.0 (default)

pure P50

unchanged — matches current behaviour

0.5

midpoint of P10 and P50

moderately conservative solar input

1.0

pure P10

fully conservative — assume worst-case solar

Example: with pv_estimate = 5.0 kW and pv_estimate10 = 2.0 kW, a bias of 0.5 produces 0.5 × 2.0 + 0.5 × 5.0 = 3.5 kW as the input to the optimizer. The optimizer then schedules more grid-charge or holds more battery reserve to compensate for the lower expected solar.

To enable, pass the parameter at runtime or add it to your configuration:

curl -i -H "Content-Type:application/json" -X POST -d '{
    "weather_forecast_pv_quantile_bias": 0.5,
    "solcast_rooftop_id": "<your_system_id>",
    "solcast_api_key": "<your_secret_api_key>"
}' http://localhost:5000/action/dayahead-optim

For EMHASS’s native Solcast path, if a forecast period does not include a pv_estimate10 value, EMHASS falls back to the central pv_estimate for that period. For caller-supplied PV forecasts, the same bias semantics are available when the optional pv_power_forecast_p10 companion is supplied alongside pv_power_forecast; see Passing your own forecast data.

Note

When the Solcast cache is enabled (weather_forecast_cache: true), the blended forecast is what gets cached. Changing weather_forecast_pv_quantile_bias therefore only takes effect on the next cache refresh; to apply a new value immediately, refresh the weather-forecast cache or run with the cache disabled.

Self-tuning the bias (adaptive conformal inference)#

Choosing a good fixed value for weather_forecast_pv_quantile_bias by hand is awkward: the value that delivers a given level of caution depends on your site, the season, and any forecast post-processing, and it drifts over time. The emhass.pv_bias_calibration module turns the knob into a self-calibrating parameter using adaptive conformal inference (ACI).

You pick an interpretable target instead of a blend weight — a shortfall rate α, i.e. how often you are willing to have realised PV come in below the forecast you planned against (e.g. 0.10). Given a logged history of (P10, P50, realised PV), a one-scalar online recursion walks the bias toward the value that holds that rate:

planned_t   = bias_t * P10_t + (1 - bias_t) * P50_t
shortfall_t = 1 if realised_t < planned_t else 0
bias_{t+1}  = clip(bias_t + gamma * (shortfall_t - α), 0, 1)

A run of shortfalls pushes the bias toward P10 (more conservative); quiet stretches relax it back toward P50. The realised shortfall rate converges to α as long as the target is inside the range the P10–P50 blend can actually express, and it keeps tracking α as conditions drift, which is the ACI guarantee. The update is asymmetric by design — it ramps protection up quickly when the forecast starts over-calling and releases it slowly afterwards.

compute_pv_bias_calibration(...) is a side-effect-free recommendation engine: it returns a recommended bias plus diagnostics (the achieved shortfall rate, the feasible range, the static shortfall-vs-bias curve, and the full bias trajectory) and never changes the live forecast or the optimization. It reports the value you can then set as weather_forecast_pv_quantile_bias. It also accepts a pluggable per-step score so a cost-aware objective (for example a threshold-weighted CRPS, which prices how much conservatism costs on clear days) can replace the plain shortfall indicator without changing the recursion.

The supported pv-bias-calibration action exposes that same engine for caller-supplied history. It is reporting-only: it does not retrieve Home Assistant history, run an optimization, write configuration, or apply the returned bias.

curl -i -H 'Content-Type:application/json' -X POST \
  -d '{
    "p10": [1.8, 2.1, 2.4, 2.0],
    "p50": [2.5, 2.9, 3.1, 2.8],
    "actual": [2.2, 2.0, 2.7, 2.3],
    "target_shortfall_rate": 0.10,
    "gamma": 0.05,
    "bias0": 0.0
  }' \
  http://localhost:5000/action/pv-bias-calibration

p10, p50, and actual are required equal-length ordered histories. Optional action fields map directly to the existing engine: curtailed, curtailment_margin, target_shortfall_rate, gamma, and bias0. The JSON response is the engine’s existing result unchanged, including recommended_bias, target_feasible, feasible_shortfall_range, achieved shortfall rates, converged, observation/exclusion counts, the bias trajectory, and the static shortfall curve.

An infeasible target is a valid diagnostic result, not an error. For example, if realised PV falls below even P10 too often for a 10% shortfall target to be expressed by any P10-P50 convex blend, target_feasible is false, converged remains false, and feasible_shortfall_range reports the achievable range honestly.

Note

The calibration action consumes history supplied by the caller; it does not automatically log P10/P50 versus realised PV. Applying recommended_bias to weather_forecast_pv_quantile_bias remains a separate, explicit user-authorised action.

Curtailment. If your system curtails (export limiting, inverter clipping), the realised PV you log is capped below what the array could have produced. A plan that was actually met then looks like a shortfall, and the tuner pushes the bias up for the wrong reason — the same contamination that is already excluded from the adjusted-PV training data (see Adjusting PV Forecasts using machine learning). Flag those steps with the curtailed argument and they are dropped from the recursion; you can pass either a boolean mask or the published curtailment power series (custom_pv_curtailment_id, by default sensor.p_pv_curtailment), which is thresholded at > 0 just as the adjusted-PV path does. Prefer available (pre-curtailment) PV over metered export where your inverter reports it. On raw timesteps, curtailment_margin=1 also drops the neighbouring steps to absorb execution lag, matching the adjusted-PV behaviour; it defaults to 0 here because a step in this history is often a whole day, where a margin would discard two good days per curtailed one. The result reports n_curtailed_excluded, and warns when so much was dropped that the recommendation only describes your non-curtailed operating regime.

The flag must come from the curtailment entity, never from the forecast error itself: a rule like “realised came in far below forecast, so it must have been curtailed” would delete exactly the shortfall observations the tuner counts and drive the bias to zero.

References

  • I. Gibbs and E. Candès (2021), Adaptive Conformal Inference Under Distribution Shift, NeurIPS 2021, arXiv:2106.00170 — the fixed-gamma recursion used here.

  • I. Gibbs and E. Candès (2024), Conformal Inference for Online Prediction with Arbitrary Distribution Shifts, JMLR, arXiv:2208.08401 — the parameter-free (DtACI) refinement that removes the gamma choice, noted as a future extension.

solar.forecast#

A third method uses the Solar.Forecast service. You will need to set method=solar.forecast and use just one parameter solar_forecast_kwp (the peak power in kW of one PV plane, which is the whole installation if you have a single plane; see below for several planes). It can be passed at runtime or set in the secrets file. This will be using the free public Solar.Forecast account with 12 API requests per hour, per IP, and 1h data resolution. As with Solcast, there are paid account services that may result in better forecasts. EMHASS interpolates between the readings Solar.Forecast returns, but only across gaps of up to one hour. Time with no reading for longer, such as the night between the last and the first reading of a day, is set to 0 W PV and not interpolated. Solar.Forecast brackets each night with 0 W sunrise and sunset readings, so a gap of more than one hour next to a non-zero reading is treated as missing data: EMHASS logs a warning with the number of affected steps and the first and last of them, and uses 0 W PV for them. This is a conservative modelling choice, not an accurate substitute for the missing production: it can under-forecast PV. Time before the first or after the last reading is also 0 W, without a warning.

For example, for a 5 kW installation:

curl -i -H "Content-Type:application/json" -X POST -d '{
	"solar_forecast_kwp":5
}' http://localhost:5000/action/dayahead-optim

If your installation has several planes (several entries in pv_module_model), EMHASS sends one request per plane, using that plane’s surface_tilt and surface_azimuth, and adds the answers up. A single number in solar_forecast_kwp is sent for every plane, so it should then be the size of one plane, not of the whole installation. If the planes differ in size, give a list with one kWp value per plane, in the order of pv_module_model:

# Two planes, 3.5 kW and 4 kW
curl -i -H "Content-Type:application/json" -X POST -d '{
	"solar_forecast_kwp":[3.5, 4]
}' http://localhost:5000/action/dayahead-optim

Every entry has to be a number above zero and the list needs exactly one entry per plane. When EMHASS fetches the forecast, anything else stops it with an error naming solar_forecast_kwp, before a request is sent or a cache file is read. The list works from the secrets file and as a runtime parameter. The add-on options page only takes a single number for now.

Note

If you use the Solcast method, or explicitly pass the PV power forecast values (see below), the pv_module_model and pv_inverter_model parameters defined in the configuration will be ignored. The Solar.Forecast method does not use the module and inverter models either, but it takes the number of planes from pv_module_model and reads surface_tilt and surface_azimuth for each plane.

PV plants and the per-plant series#

When your PV forecast comes from more than one source of production, EMHASS keeps each one as a separate “plant” next to the total. What a plant is depends on the method:

Method

A plant is

Order

open-meteo (and the deprecated scrapper)

one entry of pv_module_model (with the matching entries of the other PV lists)

the order of pv_module_model

solar.forecast

one plane, that is one entry of pv_module_model

the order of pv_module_model

solcast

one rooftop ID of solcast_rooftop_id

the order the IDs are written in (separated by commas or spaces)

csv, list or a runtime pv_power_forecast

the one series you pass (with one inverter)

not applicable

With Solcast the plants are the rooftops. With one inverter they are not tied to pv_module_model: the first rooftop ID is plant 0 whatever pv_module_model says.

When the forecast has two or more plants, the optimization results get one extra column per plant, P_PV_plant0, P_PV_plant1 and so on (counting from 0), next to P_PV. They are in opt_res_latest.csv, in the plan returned by /api/v1/plan and in the power chart and table of the web UI. They are for looking at how the total splits between plants. The optimization does not use them, and no Home Assistant sensor is created for them. With a single plant (or csv or list) the results have no such columns and are the same as before.

The plant columns add up to P_PV, up to float rounding (the plan returned by /api/v1/plan rounds to 10 decimals), and none of them is negative. The one exception is above one inverter: when a PV-only inverter with compute_curtailment off cannot convert all its PV, the optimizer clips that inverter’s PV before the solve, and P_PV (and P_PV_{i}) show the clipped power while the P_PV_plant{j} columns keep the forecast as it came in, so in those steps the plant columns add up to more than P_PV. PV that a hybrid inverter spills is not a clip: P_PV still counts it and P_PV_spill_{i} shows it. The shares come from each plant’s own forecast. If P_PV is changed afterwards (in naive-mpc-optim the first step is mixed with the live PV power, and the PV forecast adjustment (see below) replaces the whole forecast), the plant columns follow the new total and keep each plant’s share. A plant that comes out slightly below zero in the model (an inverter’s night-time standby draw, for example) counts as zero. If every plant was at zero in a step where the new total is above zero, the total is shared in proportion to each plant’s total over the whole forecast, or equally if that is zero too. Where P_PV is NaN, the plant columns are NaN too. With compute_curtailment the plants split P_PV as forecast: P_PV_curtailment is not split per plant.

Plant series are only available from the weather methods that EMHASS runs itself. With one inverter they are not worked out from a csv file or a list.

Caching PV Forecast#

For the MPC users, running optimizations regularly; You may wish to cache your PV forecast results, to reuse throughout the day. Partially for those who use the free plan of Solcast, Caching can help reduce the amount of calls bellow 10 a day. Caching Forecast data will also speed up the forecast process, bypassing the need to call to the external forecast API each MPC run.

# Run weather forecast and cache results (Recommended to run this 1-10 times a day, throughout the day)
curl -i -H 'Content-Type:application/json' -X POST -d {} http://localhost:5000/action/weather-forecast-cache

# Then run your regular MPC call (E.g. every 5 minutes)
curl -i -H 'Content-Type:application/json' -X POST -d {} http://localhost:5000/action/naive-mpc-optim

EMHASS will see the saved cache file and use its data over pulling new data from the API.

The action answers with HTTP 400 when the forecast could not be produced; the reason is in the action log, which the response does not return because it is shared by all requests. Earlier versions answered 200 in these cases, so clients that relied on that status should be updated. The failures are:

  • Solcast or solar.forecast returning an HTTP error, solar.forecast staying unreachable after its retries, a missing or unusable Solcast API key or rooftop ID, or the Solcast daily call cap already being used. A network failure that Solcast or Open-Meteo raise is not one of these and still answers 500.

  • An invalid weather_forecast_method.

  • A runtime pv_power_forecast list that is rejected, for example one with a non-finite value, one that holds lists or mappings while number_of_inverters is 1, or one paired with a pv_power_forecast_p10 list of a different length. A flat list that is too short, or a value of the wrong type, is only ignored, as in the other actions, and the configured method applies.

  • With solar.forecast, a solar_forecast_kwp list that cannot be used: a wrong number of entries, or an entry that is not a number above zero.

With the csv weather forecast method there is nothing to cache, because your file is read on every run, so the action does nothing and answers 200.

weather_forecast_cache can also be provided as a runtime parameter, in an optimization, to save the forecast results to cache:

# Example of running day-ahead and optimization storing the retrieved Solcast data to cache
curl -i -H 'Content-Type:application/json' -X POST -d '{
	"weather_forecast_cache":true
}' http://localhost:5000/action/dayahead-optim

By default, if EMHASS finds a problem with the cache file, the cache will be automatically deleted. Due to the missing cache, the next optimization will run and pull data from the External API. For Solcast only, If you wish to make sure that a certain optimization will only use the cached data, (otherwise present an error) the runtime parameter weather_forecast_cache_only can be used:

# Run the weather forecast action 1-10 times a day 
curl -i -H 'Content-Type:application/json' -X POST -d {} http://localhost:5000/action/weather-forecast-cache

# Then run your regular MPC call (E.g. every 5 minutes) and make sure it only uses the Solcast cache. (do not pull from Solcast)
curl -i -H 'Content-Type:application/json' -X POST -d '{
	"weather_forecast_cache_only":true
}' http://localhost:5000/action/naive-mpc-optim

With two or more plants, the Solcast and solar.forecast cache file also holds each plant’s series next to the total. A cache that was saved by an earlier version, or for a different number of plants than you have configured now (rooftop IDs for Solcast, entries of pv_module_model for solar.forecast), is still used for the total, exactly as before. It has no plant series, so that run has no P_PV_plant columns, and one line in the log at info level says so. Nothing is deleted or requested again because of this, so no API quota is spent. With Solcast, run the weather-forecast-cache action once to save a cache with the plant series. With solar.forecast, delete weather_forecast_data.pkl in the data folder: an existing solar.forecast cache file is used even by the weather-forecast-cache action, so the next run only fetches a new one once the file is gone. The Open-Meteo cache only holds weather data and is not affected, because the plants are computed again on every run.

Caching Open-Meteo Weather Service Usage#

When you have EMHASS configured to use the Open-Meteo weather service, to minimize API calls to the service, and to provide resilience in case of transient connectivity issues, EMHASS caches successful calls to the Open-Meteo API in a cached-open-meteo-forecast-b.json file in the data directory. The JSON file contains the default 3 days of weather forecast data. This Open-Meteo cache is independent of the PV cache discussed above and will be used even when the PV cache is not enabled. By default, when the JSON file is older than 30 minutes, attempts will be made to replace it with a more recent version from the Open-Meteo weather service. It will only be replaced if this is successful. If any errors occur the older version will continue to be used until a new version can been fetched. The maximum cache age, with a default value of 30 minutes, can be configured using the open_meteo_cache_max_age setting in config.json or as a parameter in EMHASS REST API calls. The value is specified in minutes. If you want to disable caching you can specify a value of 0.

Adjusting PV Forecasts using machine learning#

EMHASS provides methods to adjust the PV power forecast using machine learning regression techniques. The adjustment process consists of two steps: training a regression model using historical PV data and then applying the trained model to correct new PV forecasts.

This functionality may help to fine-tune the PV prediction and model some local behavior of your PV production such as: tree shading, under-production due to dust/dirt, curtailment events, local micro-weather conditions, etc.

To activate this option all that is needed is to set set_use_adjusted_pv to True in the configuration.

The Model Training uses the adjust_pv_forecast_fit method in the Forecast class. This method fits a regression model to adjust the PV forecast. It uses historical forecasted and actual PV production data as training input, incorporating additional features such as time of day and solar angles. The model is trained using time-series cross-validation, with hyperparameter tuning performed via grid search. The best model is selected based on mean squared error scoring. The historical data retrieved depends on the historic_days_to_retrieve parameter in the configuration. By default, the method uses LassoRegression, but the adjusted_pv_regression_model parameter supports the following regression models (defined in machine_learning_regressor.py): ‘LinearRegression’, ‘RidgeRegression’, ‘LassoRegression’, ‘ElasticNet’, ‘KNeighborsRegressor’, ‘DecisionTreeRegressor’, ‘SVR’, ‘RandomForestRegressor’, ‘ExtraTreesRegressor’, ‘GradientBoostingRegressor’, ‘AdaBoostRegressor’, and ‘MLPRegressor’. Once the model is trained, it computes root mean squared error (RMSE) and R² metrics to assess performance. These metrics are logged for reference. If debugging is disabled, the trained model is saved for future use.

On handled model training or cached-model loading failures, EMHASS logs the condition and uses the original unadjusted PV forecast for that optimization cycle. The adjustment layer therefore does not replace an otherwise valid base PV forecast with an invalid value.

When compute_curtailment is enabled, timesteps where PV production was curtailed are excluded from the training data, with a one-timestep margin on either side to absorb execution lag. The curtailment information is read from the history of the published curtailment entity (custom_pv_curtailment_id, by default sensor.p_pv_curtailment). During curtailment the measured production is deliberately below the achievable PV power, so training on those samples would teach the model a systematic downward bias. If the curtailment entity has no recorded history, the model is trained on unfiltered data.

The actual Forecast Adjustment is performed by the adjust_pv_forecast_predict method. This method applies the trained regression model to adjust PV forecast data. Before making predictions, the method enhances the data by adding date-based and solar-related features. It then uses the trained model to predict the adjusted forecast. A correction is applied based on solar elevation to prevent negative or unrealistic values, ensuring that the adjusted forecast remains physically meaningful. The correction based on solar elevation can be parametrized using a threshold value with parameter adjusted_pv_solar_elevation_threshold from the configuration.

Model Caching for Performance Optimization#

To improve performance and reduce unnecessary Home Assistant API calls, EMHASS implements a model caching mechanism for the adjusted PV forecast regression model. Instead of re-training the model on every optimization run, EMHASS saves the trained model to disk and reuses it if it’s still fresh.

The caching behavior is controlled by the adjusted_pv_model_max_age parameter in the optimization configuration:

  • Default value: 24 hours - The model will be re-fitted if it’s older than 24 hours

  • Set to 0: Forces re-fitting on every call (preserves original behavior without caching)

  • Custom values: Set any value in hours based on your needs (e.g., 6, 12, 48, etc.)

Runtime parameter override:

You can also override the adjusted_pv_model_max_age parameter at runtime using the API:

curl -i -H "Content-Type: application/json" -X POST -d '{
	"adjusted_pv_model_max_age": 6,
	"pv_power_forecast": [0, 0, 50, 150, ...]
}' http://localhost:5000/action/naive-mpc-optim

Load power forecast#

Note

New in EMHASS v0.12.0: the default method for load power forecast is the typical statistics-based method!

The default method for load forecast is the typical method, which uses basic statistics and a year long load power data grouped by the current day-of-the-week of the current month. This provides a typical daily load power characteristic with a 30 minute resolution. The load power is scaled using the parameter maximum_power_from_grid. This method uses the default data with 1-year of load power consumption in file data/data_train_load_clustering.pkl. You can customize this data to your own household consumption by erasing the previous file and running the script scripts/load_clustering.py (this will try to fetch 365 days of data from your load power sensor). However, if you have a working configuration without any problems with data retrieve from Home Assistant, then it is adviced to use the more advanced method mlforecaster.

A second method is a naive method, also called persistence. This is obtained using method=naive. This method simply assumes that the load at each future time step will be equal to the load observed at the same time of day on the most recent day for which that time is already known: for a forecast starting now, the next 24 hours repeat the last 24 hours, and a horizon longer than one day repeats that same last day again. The alignment is on the time of day, whatever the length of the forecast horizon (e.g. a naive-mpc-optim prediction_horizon shorter or longer than one day). Calendar days are used, so across a daylight saving time change 07:00 is still forecast from 07:00. If a time of day is missing in the history (a gap, or a time that was skipped or repeated by a daylight saving time change), it is taken from the day before. The history retrieved for this method covers delta_forecast_daily days, whose default is 1 local calendar day rather than a fixed 24-hour duration; this is enough as only the most recent day is needed.

Note

Forecast windows follow the configured local timezone. A one-day window therefore normally contains 24 hours, but a valid window crossing daylight saving time can contain 23 hours on spring-forward or 25 hours on fall-back. If the nominal calendar-day endpoint itself is in a skipped/repeated wall-clock interval, EMHASS moves a nonexistent endpoint forward by the timezone’s actual DST gap and chooses the post-transition occurrence for an ambiguous endpoint. The generated optimization timestep count follows the resolved real timeline; callers supplying external forecast lists must provide enough rows for that resolved window or an explicit shorter prediction_horizon.

This is presented graphically here:

Starting with v0.4.0, a new forecast framework is proposed within EMHASS. It provides a more efficient way to forecast the power load consumption. It is based on the skforecast module that uses scikit-learn regression models considering auto-regression lags as features. The hyperparameter optimization is proposed using Bayesian optimization from the optuna module. To use this change to method=mlforecaster in the configuration.

The API provides fit, predict and tune methods.

The following is an example of a trained model using a KNN regressor:

The naive persistence model performs very well on the 2-day test period, however, is well outperformed by the KNN regressor when back-testing on the complete training set (10 months of 30-minute time step data).

The hyperparameter tuning using Bayesian optimization improves the bare KNN regressor from \(R^2=0.59\) to \(R^2=0.75\). The optimized number of lags is \(48\).

See the machine learning forecaster section for more details.

Load forecast calibration#

With three load forecast methods available (naive, typical and mlforecaster), it is not always obvious which one tracks your own consumption best. The forecast-calibration action produces an on-demand accuracy report to help you pick. It is reporting only: it does not change any optimization and it saves no model or file.

The action retrieves your recent load history (90 days by default, since the typical method needs a long window to be meaningful), splits it chronologically into three windows (train / test / val) and scores every method with a day-ahead walk-forward. To predict a given day, a method may only see history strictly before that day, so there is no look-ahead. The report is shown in the add-on web UI as a graph (actual load versus each method over the validation window) and a metrics table (MAE, RMSE, R2, MAPE, a persistence skill score and the sample count n, per method and per split).

You can launch it from the add-on web UI with the Load forecast calibration button on the advanced page, or over the REST API with an empty body:

curl -i -H 'Content-Type:application/json' -X POST -d '{}' http://localhost:5000/action/forecast-calibration

The three day windows are optional runtime parameters, so you can trade report length against how far back it looks without editing any config. They default to the values above when omitted:

  • calibration_days_to_retrieve: how much history to pull (default 90). Lower it for a quicker run or raise it to cover more of the year. If it is too short for the split below, the action returns a clear “not enough history” message instead of a report.

  • calibration_test_days: size of the held-out test window in days (default 14).

  • calibration_val_days: size of the held-out val window in days (default 14).

curl -i -H 'Content-Type:application/json' -X POST \
  -d '{"calibration_days_to_retrieve": 120, "calibration_test_days": 21, "calibration_val_days": 21}' \
  http://localhost:5000/action/forecast-calibration

A few things to keep in mind when reading the report:

  • The naive and typical methods have no fitting step, so their train column is reported as N/A. The train column is an in-sample score for mlforecaster only.

  • The skill score is 1 - method_MAE / naive_MAE, computed over the days a method and naive both cover so the comparison is fair even when a method covers fewer days. naive is the baseline at 0 and a positive value means the method beats naive persistence.

  • mlforecaster is fitted fresh in memory on the train window for this report using a fast LinearRegression baseline; it never reads a model you previously trained with forecast-model-fit, so the report works even if you have never run a fit (and it will not stall on a slow configured estimator).

  • typical here is derived from the retrieved history window rather than the long-term typical profile used in production, and it needs prior same-month, same-weekday history, so on a short window it may cover fewer days than the other methods (compare the n columns).

  • The report’s history starts at the load sensor’s first recorded (non-missing) value, and a recorded 0 W can mark that start. If the retrieval window begins before the sensor had any data (for example a recently added sensor), that earlier stretch is left out rather than turned into 0 W history, so the report may cover fewer days than requested or return the “not enough history” message. From the first recorded value onwards, the history is cleaned with your usual set_zero_min, sensor_replace_zero and sensor_linear_interp settings. This only affects the calibration report; the history preparation used by the optimizations is unchanged.

Load cost forecast#

The default method for load cost forecast is defined for a peak and non-peak hours contract type. This is obtained using method=hp_hc_periods.

Per-deferrable-load cost overrides (hybrid heating)#

By default, every deferrable load is priced at the shared load_cost_forecast (the electricity tariff). For hybrid heating setups - where one deferrable load runs on a different commodity than another (e.g. a heat pump on electricity and a gas boiler on gas) - the shared cost track is wrong: it would dispatch both loads as if they cost the same per kWh.

To handle this, EMHASS supports an optional cost_forecast_per_deferrable_load parameter in optim_conf. It’s a list with one entry per deferrable load:

  • null (or None) for a load → keep the shared load_cost_forecast (default).

  • A list of floats (length equal to the forecast horizon, units Currency/kWh) → price that load at its own per-timestep rate.

Example: heat pump on deferrable0 keeps the electricity tariff; gas boiler on deferrable1 is priced against a flat gas rate of 0.085 €/kWh:

{
  "cost_forecast_per_deferrable_load": [
    null,
    [0.085, 0.085, 0.085, ...]
  ]
}

The objective adds an adjustment term (per_load_cost - load_cost) * p_deferrable[k] for each load with an override. When the override is unset the adjustment is identically zero, so existing setups are unaffected.

Note: this parameter only adjusts the cost in the objective; it does NOT remove a non-electric load from the electric power-balance constraint. For a gas boiler load, configure nominal_power_of_deferrable_loads[k] in the gas-input units you want the optimizer to dispatch, and exclude that load from the household electric load_forecast so the grid balance reflects only true electric demand.

When using this method you can provide a list of peak-hour periods, so you can add as many peak-hour periods as possible.

As an example for a two peak-hour periods contract you will need to define the following list in the configuration file:

- load_peak_hour_periods:
    - period_hp_1:
        - start: '02:54'
        - end: '15:24'
    - period_hp_2:
        - start: '17:24'
        - end: '20:24'
- load_peak_hours_cost: 0.1907
- load_offpeak_hours_cost: 0.1419

This example is presented graphically here:

Tariff schedule time zone#

By default the start and end times of load_peak_hour_periods are wall-clock times in the site time_zone, the same clock as the optimization timestamps. Some tariffs publish their periods on a different clock, for example a fixed UTC+10 all year while the site’s civil time moves to UTC+11 in summer. For such a tariff the same wall-clock period lands on the wrong absolute intervals during daylight saving.

The optional tariff_schedule_time_zone parameter (in optim_conf, next to load_peak_hour_periods) names the time zone identifier the periods are written in:

  • Empty or unset (the default): the periods are interpreted in the site time_zone, exactly as before.

  • A time zone identifier: each optimization timestep is re-expressed in that zone to decide whether it falls inside a peak period. The optimization timestamps themselves, the forecast days and the global time_zone are unchanged; only the timesteps selected as peak differ.

  • An identifier that is not recognised is an error: the run is aborted and the invalid value is logged. It never falls back to the site time zone.

  • It only applies to load_cost_forecast_method=hp_hc_periods. Load and production prices passed as timestamped data (or read from CSV) are not affected, and capacity_charge_window is still built by the caller, who is responsible for the clock it is written in.

Only time zone identifiers are accepted; there is no separate offset syntax. A fixed offset is available as an Etc/GMT identifier, whose sign is inverted: Etc/GMT-10 is UTC+10.

Example: a tariff with a 17:00-21:00 peak on a fixed UTC+10 clock, at a site in Australia/Sydney:

- time_zone: Australia/Sydney
- tariff_schedule_time_zone: Etc/GMT-10
- load_peak_hour_periods:
    - period_hp_1:
        - start: '17:00'
        - end: '21:00'

Date

Site clock

Peak timesteps (site time)

Standard time (UTC+10)

AEST

17:00-21:00

Daylight time (UTC+11)

AEDT

18:00-22:00

In both cases the peak is the same absolute interval, 07:00-11:00 UTC. Without tariff_schedule_time_zone the peak would stay at 17:00-21:00 site time during daylight time.

PV production selling price forecast#

The default method for this forecast is simply a constant value. This can be obtained using method=constant.

Then you will need to define the photovoltaic_production_sell_price variable to provide the correct price for energy injected to the grid from excess PV production in currency/kWh.

Passing your own forecast data#

For all the needed forecasts in EMHASS, two other methods allow the user to provide their own forecast value. This may be used to provide a forecast provided by a more powerful and accurate forecaster. The two methods are: csv and list.

For the csv method you should push a csv file to the data folder. The CSV file should contain no header and the timestamped data should have the following format:

2021-04-29 00:00:00+00:00,287.07
2021-04-29 00:30:00+00:00,274.27
2021-04-29 01:00:00+00:00,243.38
...

Each timestamp is read as an absolute instant: include the UTC offset (a timestamp without one is read as UTC). A file written in local time may therefore change offset in the middle, for example +01:00 and +02:00 across a daylight saving change, and both occurrences of a repeated local hour stay distinct. The rows should cover the whole optimization horizon at the optimization_time_step. For a step more than one source step (the median row spacing) after the last row or before the first, EMHASS keeps using the closest row and logs a warning (not checked for a file with a single row) with the number of uncovered steps and the first and last of them.

For the list method, you just have to add the data as a list of values to a data dictionary during the call to emhass using the runtimeparams option.

The possible dictionary keys to pass data are:

  • pv_power_forecast for the PV power production forecast.

  • load_power_forecast for the Load power forecast.

  • load_cost_forecast for the Load cost forecast.

  • prod_price_forecast for the PV production selling price forecast.

For example, if using the add-on or the docker method, you can pass this data as a list of values to the data dictionary during the curl POST:

curl -i -H "Content-Type: application/json" -X POST -d '{
	"pv_power_forecast":[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 70, 141.22, 246.18, 513.5, 753.27, 1049.89, 1797.93, 1697.3, 3078.93, 1164.33, 1046.68, 1559.1, 2091.26, 1556.76, 1166.73, 1516.63, 1391.13, 1720.13, 820.75, 804.41, 251.63, 79.25, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
}' http://localhost:5000/action/dayahead-optim

You need to be careful here to send the correct amount of data on this list, the correct length. For example, if the data time step is defined as 1 hour and you are performing a day-ahead optimization, then this list should contain one value per step: 24 hourly values, or 23 or 25 across a daylight saving change.

Understanding forecast data formats: List vs Dictionary#

EMHASS supports two formats for passing forecast data as runtime parameters: list format and dictionary format with timestamps. Understanding when to use each format can significantly simplify your Home Assistant automations.

Both formats follow the Forecast input contract: every value must be a finite real number (no NaN, null, booleans or strings), prices and temperatures may be negative, and the load and PV power that reach the optimizer are kept at or above 0 W.

List Format (Traditional)#

The list format passes forecast values as a simple array. This is the most compact format but requires careful attention to timing and data length.

Requirements:

  • The list must start with the value for the first step of the optimization grid (set by method_ts_round; with first it is the step that contains now), followed by future values

  • The list must contain at least one value per horizon step based on optimization_time_step. Extra values are ignored, and a shorter list is rejected with an error rather than padded

  • Values must be in the correct order (chronological)

Example:

curl -i -H "Content-Type: application/json" -X POST -d '{
	"load_cost_forecast": [0.25, 0.24, 0.23, 0.26, 0.28, ...],
	"prediction_horizon": 24,
	"optimization_time_step": 60
}' http://localhost:5000/action/dayahead-optim

For a 24-hour forecast with 1-hour intervals, you need one value per grid step, 24 hourly values (23 or 25 across a daylight saving change), starting at the first grid step.

When to use list format:

  • Your forecast data is already aligned with EMHASS intervals

  • You want minimal JSON payload size

  • You generate forecast values programmatically in the correct interval

Dictionary Format with Timestamps (Recommended for most users)

The dictionary format allows you to pass forecast values with explicit timestamps. EMHASS will automatically handle aggregation, alignment, and gap filling.

Advantages:

  • Automatic alignment: If your data is hourly but EMHASS runs at 15-minute intervals, each hourly value is held for the 15-minute steps it covers

  • Flexible timing: No need to calculate exact list length or worry about current time alignment

  • Self-documenting: Timestamps make it clear which value applies when

  • Automatic gap filling: Each value holds until the next supplied point, and steps before the first point take the first value

  • Timezone aware: Timestamps are parsed and converted to your configured timezone

Format:

{
	"load_cost_forecast": {
		"2025-10-16 10:00:00+00:00": 0.25,
		"2025-10-16 11:00:00+00:00": 0.28,
		"2025-10-16 12:00:00+00:00": 0.23
	}
}

Example curl command:

curl -i -H "Content-Type: application/json" -X POST -d '{
	"load_cost_forecast": {
		"2025-10-16 10:00:00+00:00": 0.25,
		"2025-10-16 11:00:00+00:00": 0.28,
		"2025-10-16 12:00:00+00:00": 0.23
	},
	"prediction_horizon": 48,
	"optimization_time_step": 15
}' http://localhost:5000/action/naive-mpc-optim

What happens internally: the values are validated, aggregated to the optimization step and matched to the grid by UTC instant, exactly as described under Forecast input contract.

When to use dictionary format:

  • Your price data comes from an API (Nordpool, ENTSO-E, Amber, etc.) that provides hourly prices

  • You want EMHASS to handle interval conversion automatically

  • Your optimization_time_step differs from your data resolution

  • You want more readable and maintainable Home Assistant templates

Practical Home Assistant Examples

Dictionary format with Nordpool (Simple - Recommended!):

shell_command:
  emhass_dayahead_nordpool: >
    {% set prices = (state_attr("sensor.nordpool_kwh_be_eur_3_10_025", "raw_today") or []) + (state_attr("sensor.nordpool_kwh_be_eur_3_10_025", "raw_tomorrow") or []) %}
    {% set mapping = dict(zip(prices | map(attribute="start") | map("string") | list, prices | map(attribute="value") | list)) %}
    curl -i -H "Content-Type: application/json" -X POST -d '{
      "load_cost_forecast": {{ mapping | tojson }},
      "prod_price_forecast": {{ mapping | tojson }}
    }' http://localhost:5000/action/dayahead-optim

raw_today is a list of {start, end, value} objects, not the {timestamp: value} mapping EMHASS expects (passing it with | tojson as is makes the optimisation fail), so the template pairs each start with its value. Prices are placed by timestamp, so this is correct for hourly and 15-minute prices and on daylight saving days. raw_today + raw_tomorrow covers the whole day-ahead horizon. The value must be in currency per kWh, as for every EMHASS price (divide a per-MWh sensor by 1000).

In Home Assistant templates zip is a function, zip(keys, values), not a filter (keys | zip(values) fails with “No filter named ‘zip’”); dict(...) and tojson are available. The or [] keeps the template from failing when an attribute is missing, but then it renders an empty mapping ({}) or only today’s prices, which does not make a valid day-ahead request: guard the automation, as described next. The start values are timestamps with their UTC offset (2026-10-03 00:00:00+10:00), which EMHASS reads as absolute instants.

The tojson filter of Home Assistant writes an apostrophe as ', so a value or timestamp containing ' cannot end the single-quoted -d argument (checked with the Home Assistant template engine: the rendered command stays one -d argument and the payload valid JSON); EMHASS then rejects a non-numeric price.

Run it only when the prices cover the whole forecast window. A mapping that starts after now (today’s prices missing), ends before the window does (tomorrow’s not published, or an unavailable sensor that kept old attributes) or has a gap is silently completed by EMHASS: steps before the first point take its price and steps after the last point hold the last price, so the plan is valid but wrongly priced. The window of a day-ahead run is one local calendar day from now, so check coverage from the timestamps, not from a count of entries (a day has 23, 24 or 25 hourly entries, and 92, 96 or 100 quarter-hour entries). As an automation condition:

condition:
  - condition: template
    value_template: >
      {%- set s = "sensor.nordpool_kwh_be_eur_3_10_025" -%}
      {%- set p = (state_attr(s, "raw_today") or []) + (state_attr(s, "raw_tomorrow") or []) -%}
      {%- set ns = namespace(ok = p | length > 0) -%}
      {%- for i in range(1, p | length) -%}
        {%- if as_timestamp(p[i].start) != as_timestamp(p[i - 1].end) -%}{%- set ns.ok = false -%}{%- endif -%}
      {%- endfor -%}
      {{ states(s) not in ['unknown', 'unavailable'] and ns.ok
         and as_timestamp(p[0].start) <= as_timestamp(now())
         and as_timestamp(p[-1].end) >= as_timestamp(now() + timedelta(days=1)) }}

It requires the first entry to start at or before now, the entries to be contiguous, and the last one to end at or after the same wall-clock time tomorrow (now() + timedelta(days=1) is a local calendar day, as in EMHASS). With a delta_forecast_daily other than 1, replace 1 by that number of days.

List format with Nordpool (Traditional - More Complex):

shell_command:
  emhass_dayahead_nordpool_list: >
    curl -i -H "Content-Type: application/json" -X POST -d '{
      "load_cost_forecast": {{(
        (state_attr("sensor.nordpool_kwh_be_eur_3_10_025", "raw_today") | map(attribute="value") | list +
        state_attr("sensor.nordpool_kwh_be_eur_3_10_025", "raw_tomorrow") | map(attribute="value") | list)
        [now().hour:][:24]
      )}}
    }' http://localhost:5000/action/dayahead-optim

Notice how the list format requires:

  • Extracting values from dictionaries map(attribute="value")

  • Slicing from current hour [now().hour:]. With a 60-minute step this is the first grid step only for method_ts_round: first. With the default nearest from about :30 on (an exact :30 goes to the even UTC hour), and with last (a time that is not exactly on a step), the grid starts one hour later, so every value is applied one hour too late

  • Limiting to the correct length [:24]

raw_today already starts with the first hour of the local day, so [now().hour:] alone selects the current hour; do not prepend another value, which would shift every price one hour early. This positional list is right only for hourly prices on a day with 24 entries: with 15-minute prices (96 entries) or on a daylight saving day (its raw_today has 23 or 25 entries, so [now().hour:] is one hour off after the change) it selects the wrong prices, so use the dictionary format there (or always). A missing attribute does not stop this template from rendering (Home Assistant treats it as empty), and a list gives no protection against missing prices: with raw_today missing and raw_tomorrow present, the slice takes tomorrow’s prices as today’s, and in the first hour of the day it is still long enough, so EMHASS accepts a list of the wrong day. Use the same coverage condition as for the dictionary format, or better the dictionary format itself.

Dictionary format handles all of this automatically!

Important Notes

Timezone handling:

  • Timestamps should include timezone information (e.g., +00:00 for UTC, +02:00 for CEST)

  • EMHASS will convert timestamps to your configured timezone from secrets_emhass.yaml

  • If timestamps don’t include timezone, they’re read as UTC, not as your local time. Always include the offset to avoid a silent shift

Data resolution:

  • Dictionary format works best when your source data is at a coarser resolution than your optimization_time_step

  • Example: Hourly price data automatically resampled to 15-minute intervals

  • Each hourly price is held constant for all 15-minute intervals within that hour (step semantics, no interpolation)

Current values:

  • When using list format, the first value in the list must be for the first grid step: with method_ts_round: first the step that contains now, with last the first step boundary not before now, with nearest (default) the closest boundary

  • Example: at 14:50 with 30-minute intervals, nearest starts the grid at 15:00

  • Dictionary format automatically handles this by using timestamps

Mixing formats:

  • You can use list format for some forecasts and dictionary format for others in the same API call

  • Example: pv_power_forecast as list, load_cost_forecast as dictionary

curl -i -H "Content-Type: application/json" -X POST -d '{
	"pv_power_forecast": [0, 0, 50, 150, 300, ...],
	"load_cost_forecast": {
		"2025-10-16 10:00:00+00:00": 0.25,
		"2025-10-16 11:00:00+00:00": 0.28
	}
}' http://localhost:5000/action/naive-mpc-optim

Summary: Which Format Should I Use?

Scenario

Recommended Format

Reason

Using Nordpool/ENTSO-E/Amber price APIs

Dictionary

Data already includes timestamps

Data from Home Assistant sensors with attributes

Dictionary

Easier template syntax

Hourly data with 15-min optimization intervals

Dictionary

Automatic resampling

Self-generated forecast in Python/Node-RED

Either

Choose based on convenience

Minimal network payload needed

List

More compact JSON

Generated forecast already matches intervals

List

No conversion needed

For most Home Assistant users: Use dictionary format with timestamps for simpler, more maintainable automations.

Example using: Solcast forecast + Amber prices#

If you’re using Solcast then you can define the following sensors in your system:

sensors:

  - platform: rest
    name: "Solcast Forecast Data"
    json_attributes:
      - forecasts
    resource: https://api.solcast.com.au/rooftop_sites/yyyy/forecasts?format=json&api_key=xxx&hours=24
    method: GET
    value_template: "{{ (value_json.forecasts[0].pv_estimate)|round(2) }}"
    unit_of_measurement: "kW"
    device_class: power
    scan_interval: 8000
    force_update: true

  - platform: template
    sensors:
      solcast_24hrs_forecast :
        value_template: >-
          {%- set power = state_attr('sensor.solcast_forecast_data', 'forecasts') | map(attribute='pv_estimate') | list %}
          {%- set values_all = namespace(all=[]) %}
          {% for i in range(power | length) %}
           {%- set v = (power[i] | float |multiply(1000) ) | int(0) %}
            {%- set values_all.all = values_all.all + [ v ] %}
          {%- endfor %} {{ (values_all.all)[:48] }}

With this, you can now feed this Solcast forecast to EMHASS along with the mapping of the Amber prices.

An MPC call may look like this for 4 deferrable loads:

    post_mpc_optim_solcast: "curl -i -H \"Content-Type: application/json\" -X POST -d '{\"load_cost_forecast\":{{(
          ([states('sensor.amber_general_price')|float] +
          state_attr('sensor.amber_general_forecast', 'forecasts') |map(attribute='per_kwh')|list)[:48])
          }}, \"prod_price_forecast\":{{(
          ([states('sensor.amber_feed_in_price')|float] +
          state_attr('sensor.amber_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list)[:48]) 
          }}, \"pv_power_forecast\":{{states('sensor.solcast_24hrs_forecast')
          }}, \"prediction_horizon\":48,\"soc_init\":{{(states('sensor.powerwall_charge')|float(0))/100
          }},\"soc_final\":0.05,\"operating_hours_of_each_deferrable_load\":[2,0,0,0]}' http://localhost:5000/action/naive-mpc-optim"

Thanks to @purcell_labs for this example configuration.

Note

This example sends plain lists, so every list is positional from the first optimization step (see Step indices and other positional inputs), and it prepends the current price as that first value. That only matches the grid when the first step is the 30-minute step containing now (method_ts_round: first) and the Amber forecast rows continue directly after the current interval. With the default nearest, a call in the second half of a step starts one step later and every price lands one step early. Amber forecast rows carry their own start_time, so a timestamped mapping avoids the alignment question.

Example combining multiple Solcast configurations#

If you have multiple rooftops, for example for east-west facing solar panels, then you will need to fuze the sensors providing the different forecasts on a single one using templates in Home Assistant. Then feed that single sensor data passing the data as a list when calling the shell command.

Here is a sample configuration to achieve this, thanks to @gieljnssns for sharing.

The two sensors using rest sensors:

- platform: rest
  name: "Solcast Forecast huis"
  json_attributes:
    - forecasts
  resource: https://api.solcast.com.au/rooftop_sites/xxxxxxxxxxc/forecasts?format=json&api_key=xxxxxxxxx&hours=24
  method: GET
  value_template: "{{ (value_json.forecasts[0].pv_estimate)|round(2) }}"
  unit_of_measurement: "kW"
  device_class: power
  scan_interval: 86400
  force_update: true

- platform: rest
  name: "Solcast Forecast garage"
  json_attributes:
    - forecasts
  resource: https://api.solcast.com.au/rooftop_sites/xxxxxxxxxxc/forecasts?format=json&api_key=xxxxxxxxx&hours=24
  method: GET
  value_template: "{{ (value_json.forecasts[0].pv_estimate)|round(2) }}"
  unit_of_measurement: "kW"
  device_class: power
  scan_interval: 86400
  force_update: true

Then two templates, one for each sensor:

    solcast_24hrs_forecast_garage:
      value_template: >-
        {%- set power = state_attr('sensor.solcast_forecast_garage', 'forecasts') | map(attribute='pv_estimate') | list %}
        {%- set values_all = namespace(all=[]) %}
        {% for i in range(power | length) %}
          {%- set v = (power[i] | float |multiply(1000) ) | int(0) %}
          {%- set values_all.all = values_all.all + [ v ] %}
        {%- endfor %} {{ (values_all.all)[:48] }}

    solcast_24hrs_forecast_huis:
      value_template: >-
        {%- set power = state_attr('sensor.solcast_forecast_huis', 'forecasts') | map(attribute='pv_estimate') | list %}
        {%- set values_all = namespace(all=[]) %}
        {% for i in range(power | length) %}
          {%- set v = (power[i] | float |multiply(1000) ) | int(0) %}
          {%- set values_all.all = values_all.all + [ v ] %}
        {%- endfor %} {{ (values_all.all)[:48] }}

And the fusion of the two sensors:

    solcast_24hrs_forecast:
      value_template: >-
        {% set a = states("sensor.solcast_24hrs_forecast_garage")[1:-1].split(',') | map('int') | list %}
        {% set b = states("sensor.solcast_24hrs_forecast_huis")[1:-1].split(',') | map('int') | list %}
        {% set ns = namespace(items = []) %}
        {% for i in range(a | length) %}
          {% set ns.items = ns.items + [ a[i]  + b[i]  ] %}
        {% endfor %}
        {{ ns.items }}

And finally the shell command:

dayahead_optim: "curl -i -H \"Content-Type:application/json\" -X POST -d '{\"pv_power_forecast\":{{states('sensor.solcast_24hrs_forecast')}}}' http://localhost:5001/action/dayahead-optim"

Example using the Nordpool integration#

The Nordpool integration provides spot market electricity prices (consumption and production) for the Nordic, Baltic and part of Western Europe. An integration for Home Assistant can be found here: https://github.com/custom-components/nordpool

After setup the sensors should appear in Home Assistant for raw today and tomorrow values.

The subsequent shell command to concatenate today and tomorrow values can be for example:

shell_command:
  trigger_nordpool_forecast: "curl -i -H \"Content-Type: application/json\" -X POST -d '{\"load_cost_forecast\":{{((state_attr('sensor.nordpool', 'raw_today') | map(attribute='value') | list  + state_attr('sensor.nordpool', 'raw_tomorrow') | map(attribute='value') | list))[now().hour:][:24] }},\"prod_price_forecast\":{{((state_attr('sensor.nordpool', 'raw_today') | map(attribute='value') | list  + state_attr('sensor.nordpool', 'raw_tomorrow') | map(attribute='value') | list))[now().hour:][:24]}}}' http://localhost:5000/action/dayahead-optim"

This command assumes a 60-minute optimization_time_step and method_ts_round: first, so that the list starts at the current hour (see the list format). With another step or rounding mode, prefer the dictionary format.

Now/current values in forecasts#

When implementing MPC applications with high optimization_time_step it can be interesting if, at each MPC iteration, the forecast values are updated with the real now/current values measured from live data. This is useful to improve the accuracy of the short-term forecasts. As shown in some of the references below, mixing with a persistence model makes sense since this type of model performs very well at low temporal resolutions (intra-hour).

A simple integration of current/now values for PV and load forecast is implemented using a mixed one-observation persistence model and the one-step-ahead forecasted values from the current passed method.

This can be represented by the following equation at time \(t=k\):

\[P^{mix}_{PV} = \alpha \hat{P}_{PV}(k) + \beta P_{PV}(k-1)\]

Where \(P^{mix}_{PV}\) is the mixed power forecast for PV production, \(\hat{P}_{PV}(k)\) is the current first element of the original forecast data, \(P_{PV}(k-1)\) is the now/current value of PV production and \(\alpha\) and \(\beta\) are coefficients that can be fixed to reflect desired dominance of now/current values over the original forecast data or vice-versa.

The alpha and beta values can be passed in the dictionary using the runtimeparams option during the call to emhass. If not passed they will both take the default 0.5 value. These values should be fixed following your own analysis of how much weight you want to put on measured values to be used as the persistence forecast. This will also depend on your fixed optimization time step. As a default, they will be at 0.5, but if you want to give more weight to measured persistence values, then you can try lower \(\alpha\) and rising \(\beta\), for example: alpha=0.25, beta=0.75. After this, you will need to check with the recorded history if these values fit your needs.

References#

  • E. Lorenz, J. Kuhnert, A. Hammer, D. Heinemann, Photovoltaic (PV) power predictions with PV measurements, satellite data and numerical weather predictions. Presented at CM2E, Energy & Environment Symposium, Martinique, 2014.

  • Maimouna Diagne, Mathieu David, Philippe Lauret, John Boland, NicolasSchmutz, Review of solar irradiance forecasting methods and a proposition for small-scale insular grids. Renewable and Sustainable Energy Reviews 27 (2013) 65–76.

  • Bryan Lima, Sercan O. Arik, Nicolas Loeff, Tomas Pfister, Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting. arXiv:1912.09363v3 [stat.ML] 27 Sep 2020.