Datasets & Models

Dataset

The available datasets are listed in the data folder, with metadata provided in data/metadata.xlsx.

Simple initialization example: Run this in Python. On macOS, start Python with python3 if python is not available.

import pynnlf

pynnlf.init("example_project")

In this example, only the sample dataset (ds0) is included by default. Additional datasets are downloaded only if you enable downloads.

To download all datasets, use:

import pynnlf

pynnlf.init(
    "example_project",
    download_data=True,
    all_data=True,
)

To download a specific subset of datasets:

import pynnlf

pynnlf.init(
    "example_project",
    download_data=True,
    datasets=["ds14", "ds15"],
)

The download is typically fast because the dataset files are relatively small.

Below are some of the datasets currently included in the library.

Ausgrid Solar Home Datasets

This dataset has been widely used in net load forecasting research since 2016. It includes data from 300 solar-equipped households within the Ausgrid network in Sydney. The original source is here.

Dataset Name	Description
ds1_ashd.csv	Ausgrid Solar Home Dataset, aggregate of 300 household data in Ausgrid network
ds4_ashd_with_weather.csv	ds1 enhanced with temperature, relative humidity, and wind speed data
ds13_ashd_with_cloud_solcast.csv	ds4 further enriched with cloud data from Solcast
ds19_ashd_148hh.csv	ASHD aggregate of the 148 households nearest to Sydney Airport, selected by postcode-centroid distance
ds20_ashd_148hh_with_weather.csv	ds19 enhanced with Sydney Airport temperature, relative humidity, and wind speed data
ds21_ashd_148hh_with_weather_cloud_solcast.csv	ds20 further enriched with Sydney Airport Solcast cloud opacity data

Australia Energy Data Platform (AEDP) Datasets

These datasets were compiled by UNSW Sydney using data from Solar Analytics and Wattwatchers. Sensitive information such as customer addresses, names, and NMIs has been removed. The original source is here.

Dataset Name	Description
ds10_aedp_cluster2_30min.csv	AEDP dataset for Cluster 2 with 30-minute resolution
ds11_aedp_cluster2_30min_with_weather.csv	ds10 enhanced with weather data including temperature, humidity, etc.
ds25-ds36 AEDP aggregation datasets	Publication datasets for AEDP aggregation-level experiments at 1, 10, 100, and synthetic-bootstrap 1000 households, with 3 samples per level and Sydney Airport weather variables. These are generated by publication/journal_article_1/notebooks/03_aedp_aggregation_level from 139 valid per-household AEDP checkpoints; household series are forward-filled before aggregation.

Solar Analytics CICCADA BESS Datasets

These datasets were generated for the publication workflow from Solar Analytics CICCADA BESS data. The 100-household variants are retained for exploration because 56 of the 100 selected households contain meaningful negative reconstructed underlying-load intervals. The 44-household variants keep only households with no intervals below the -0.05 kW meaningful-negative threshold.

Dataset Name	Description
ds16_sa_bess_underlying_load_30min.csv	Exploratory 100-household aggregate underlying load
ds17_sa_bess_net_load_with_pv_30min.csv	Exploratory 100-household aggregate net load with PV
ds18_sa_bess_net_load_with_pv_battery_30min.csv	Exploratory 100-household aggregate net load with PV and battery
ds22_sa_bess_44hh_pos_underlying_load_30min.csv	Clean 44-household aggregate underlying load
ds23_sa_bess_44hh_pos_net_load_with_pv_30min.csv	Clean 44-household aggregate net load with PV
ds24_sa_bess_44hh_pos_net_load_with_pv_battery_30min.csv	Clean 44-household aggregate net load with PV and battery

Ausgrid Zone Substation (ZS) Datasets - Mascot

Unlike the previous household-focused datasets, this dataset covers a zone substation, which includes residential, commercial and industrial (C&I), and major customers. The original source is here.

Dataset Name	Description
ds14_ausgrid_zs_mascot.csv	Zone Substation data for Mascot
ds15_ausgrid_zs_mascot_30min_with_weather.csv	ds14 enhanced with weather data at 30-minute resolution

Model

Forecasting Models Overview

Model ID	Model Name	Short Description
m1_naive	Naive	Forecast equals the last observed value
m2_snaive	Seasonal Naive	Forecast equals the value from the same season in the previous cycle

Statistical Models

Model ID	Model Name	Short Description
m3_ets	ETS	Exponential smoothing model with error, trend, and seasonality components
m4_arima	ARIMA	Autoregressive Integrated Moving Average model for time series forecasting
m5_sarima	SARIMA	Seasonal ARIMA model with seasonal components
m6_lr	Linear Regression	Predicts future values using a linear combination of input features

Machine Learning Models

Model ID	Model Name	Short Description
m7_ann	ANN	Basic Artificial Neural Network with one hidden layer
m8_dnn	Deep Neural Network	ANN with more than one hidden layer
m9_rt	Regression Tree	Decision tree model for regression tasks
m10_rf	Random Forest	Ensemble of regression trees for improved accuracy and robustness
m11_svr	Support Vector Regression	Uses support vectors to perform regression with margin of tolerance
m12_rnn	Recurrent Neural Network	Neural network with feedback loops for sequential data
m13_lstm	Long Short-Term Memory	RNN variant designed to capture long-term dependencies
m14_gru	Gated Recurrent Unit	Simplified LSTM with fewer parameters
m15_transformer	Transformer	Attention-based model for sequence modeling without recurrence
m16_prophet	Prophet	Time series model developed by Facebook for business forecasting
m17_xgb	XGBoost	Gradient boosting framework optimized for speed and performance
m18_nbeats	N-BEATS	Deep learning model for univariate time series forecasting

Hyperparameter

The list of available models and their hyperparameters can be seen in models/hyperparameters.yaml. The values currently available are the hyperparameter values mostly used in academic literature, but not necessarily the optimum value.