Ensure solar data updates at midnight#3331
Conversation
There was a problem hiding this comment.
Pull request overview
Addresses issue #3330 by attempting to refresh Solcast/solar forecast data promptly after a day rollover (midnight) so plans calculated shortly after midnight don’t use stale “yesterday” values.
Changes:
- Add day-rollover / age-based logic in
SolarAPI.run()to triggerfetch_pv_forecast()outside the existing plan-interval cadence. - Track
last_fetched_timestampafter forecast fetch attempts. - Update chart pruning behavior and series colors in the web charts UI.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
apps/predbat/solcast.py |
Adds “new day or older than 60 minutes” fetch logic and tracks last fetch time to improve solar forecast freshness around midnight. |
apps/predbat/web.py |
Adjusts LoadMLPower chart history pruning and updates several chart series colors. |
| fetch_age = 9999 | ||
| same_day = False | ||
| if self.last_fetched_timestamp: | ||
| fetch_age = (self.now_utc_exact - self.last_fetched_timestamp).total_seconds() / 60 | ||
| same_day = self.last_fetched_timestamp.date() == self.now_utc_exact.date() | ||
|
|
||
| if seconds % (self.plan_interval_minutes * 60) == 0: # Every plan_interval_minutes | ||
| await self.fetch_pv_forecast() | ||
| elif not same_day or (fetch_age > 60): # If data is older than 60 minutes or it's a new day, fetch new data | ||
| await self.fetch_pv_forecast() |
There was a problem hiding this comment.
self.last_fetched_timestamp is set using self.now_utc_exact (timezone-aware), but run() compares it to datetime.now() (naive). Subtracting/comparing these will raise a TypeError at runtime. Use a consistent, tz-aware clock for both values (e.g., now = self.now_utc_exact or datetime.now(self.local_tz)), and avoid mixing naive/aware datetimes.
| async def run(self, seconds, first): | ||
| """ | ||
| Run the Solar API | ||
| """ | ||
| fetch_age = 9999 | ||
| same_day = False | ||
| if self.last_fetched_timestamp: | ||
| fetch_age = (self.now_utc_exact - self.last_fetched_timestamp).total_seconds() / 60 | ||
| same_day = self.last_fetched_timestamp.date() == self.now_utc_exact.date() | ||
|
|
||
| if seconds % (self.plan_interval_minutes * 60) == 0: # Every plan_interval_minutes | ||
| await self.fetch_pv_forecast() | ||
| elif not same_day or (fetch_age > 60): # If data is older than 60 minutes or it's a new day, fetch new data | ||
| await self.fetch_pv_forecast() | ||
| return True |
There was a problem hiding this comment.
The new midnight/age-based refresh logic in run() is not covered by existing tests (current tests exercise fetch_pv_forecast() directly but not the scheduler behavior). Add a unit test that simulates a day rollover and verifies run() triggers a fetch shortly after midnight (and does not refetch every minute once updated). This will also prevent regressions around timezone handling.
| @@ -2659,12 +2659,12 @@ def get_chart(self, chart): | |||
| {"name": "Load Power (Actual)", "data": load_power, "opacity": "1.0", "stroke_width": "3", "stroke_curve": "smooth", "color": "#3291a8", "unit": "kW"}, | |||
| {"name": "Load Power (ML Predicted Future)", "data": load_ml_forecast_power, "opacity": "0.5", "stroke_width": "3", "chart_type": "area", "stroke_curve": "smooth", "color": "#eb2323", "unit": "kW"}, | |||
| {"name": "Load Power (Used)", "data": load_power_best, "opacity": "1.0", "stroke_width": "2", "stroke_curve": "smooth", "unit": "kW"}, | |||
| {"name": "Load Power ML History", "data": power_today, "opacity": "1.0", "stroke_width": "2", "stroke_curve": "smooth", "unit": "kW"}, | |||
| {"name": "Load Power ML History +1h", "data": power_today_h1, "opacity": "1.0", "stroke_width": "2", "stroke_curve": "smooth", "unit": "kW"}, | |||
| {"name": "Load Power ML History +8h", "data": power_today_h8, "opacity": "1.0", "stroke_width": "2", "stroke_curve": "smooth", "unit": "kW"}, | |||
| {"name": "Load Power ML History", "data": power_today, "opacity": "1.0", "stroke_width": "2", "stroke_curve": "smooth", "unit": "kW", "color": "#eb2323"}, | |||
| {"name": "Load Power ML History +1h", "data": power_today_h1, "opacity": "1.0", "stroke_width": "2", "stroke_curve": "smooth", "unit": "kW", "color": "#716d63"}, | |||
| {"name": "Load Power ML History +8h", "data": power_today_h8, "opacity": "1.0", "stroke_width": "2", "stroke_curve": "smooth", "unit": "kW", "color": "#a6a5a3"}, | |||
| {"name": "PV Power (Actual)", "data": pv_power, "opacity": "1.0", "stroke_width": "3", "stroke_curve": "smooth", "color": "#f5c43d", "unit": "kW"}, | |||
| {"name": "PV Power (Predicted)", "data": pv_power_best, "opacity": "0.7", "stroke_width": "2", "stroke_curve": "smooth", "chart_type": "area", "color": "#ffa500", "unit": "kW"}, | |||
| {"name": "Temperature", "data": temperature_forecast, "opacity": "1.0", "stroke_width": "2", "stroke_curve": "smooth", "color": "#ff6b6b", "unit": "°C"}, | |||
| {"name": "Temperature", "data": temperature_forecast, "opacity": "1.0", "stroke_width": "2", "stroke_curve": "smooth", "color": "#75ff6b", "unit": "°C"}, | |||
There was a problem hiding this comment.
This PR is titled/described as fixing Solcast updates at midnight, but this hunk also changes LoadMLPower chart data pruning and several chart colors. If these UI/graph changes are intentional, please link them to the issue or split them into a separate PR so the bugfix can be reviewed/reverted independently.
1 participant
Uh oh!
There was an error while loading. Please reload this page.