Contents 1 Introduction to Power Market Data 1.1 Overview of Electricity Markets 1.2 Organization and Data Disclosure of Electricity Market 1.2.1 Transaction Data 1.2.2 Price Data 1.2.3 Supply and Demand Data 1.2.4 System Operation Data 1.2.5 Forecast Data 1.2.6 Confidential Data 1.3 Conclusions References PartⅠLoad Modeling and Forecasting 2 Load Forecasting with Smart Meter Data 2.1 Introduction 2.2 Framework 2.3 Ensemble Learning for Probabilistic Forecasting 2.3.1 Quantile Regression Averaging 2.3.2 Factor Quantile Regression Averaging 2.3.3 LASSO Quantile Regression Averaging 2.3.4 Quantile Gradient Boosting Regression Tree 2.3.5 Rolling Window-Based Forecasting 2.4 Case Study 2.4.1 Experimental Setups 2.4.2 Evaluation Criteria 2.4.3 Experimental Results 2.5 Conclusions References 3 Load Data Cleaning and Forecasting 3.1 Introduction 3.2 Characteristics of Load Profiles 3.2.1 Low-Rank Property of Load Profiles 3.2.2 Bad Data in Load Profiles 3.3 Methodology 3.3.1 Framework 3.3.2 Singular Value Thresholding (SVT) 3.3.3 Quantile RF Regression 3.3.4 Load Forecasting 3.4 Evaluation Criteria 3.4.1 Data Cleaning-Based Criteria 3.4.2 Load Forecasting-Based Criteria 3.5 Case Study 3.5.1 Result of Data Cleaning 3.5.2 Day Ahead Point Forecast 3.5.3 Day Ahead Probabilistic Forecast 3.6 Conclusions References 4 Monthly Electricity Consumption Forecasting 4.1 Introduction 4.2 Framework
4.2.1 Data Collection and Treatment 4.2.2 SVECM Forecasting 4.2.3 Self-adaptive Screening 4.2.4 Novelty and Characteristics of SAS-SVECM 4.3 Data Collection and Treatment 4.3.1 Data Collection and Tests 4.3.2 Seasonal Adjustments Based on X-12-ARIMA 4.4 SVECM Forecasting 4.4.1 VECM Forecasting 4.4.2 Time Series Extrapolation Forecasting 4.5 Self-adaptive Screening 4.5.1 Influential EEF Identification 4.5.2 Influential EEF Grouping 4.5.3 Forecasting Performance Evaluation Considering Different EEF Groups 4.6 Case Study 4.6.1 Basic Data and Tests 4.6.2 Electricity Consumption Forecasting Performance Without SAS 4.6.3 EC Forecasting Performance with SAS 4.6.4 SAS-SVECM Forecasting Comparisons with Other Forecasting Methods 4.7 Conclusions References 5 Probabilistic Load Forecasting 5.1 Introduction 5.2 Data and Model 5.2.1 Load Dataset Exploration 5.2.2 Linear Regression Model Considering Recency-Effects 5.3 Pre-Lasso BasedFeature Selection 5.4 Sparse PenalizedQuantileRegression (Quantile-Lasso) 5.4.1 Problem Formulation 5.4.2 ADMM Algorithm 5.5 Implementation 5.6 Case Study 5.6.1 Experiment Setups 5.6.2 Results 5.7 Concluding Remarks References Part ⅡElectricity Price Modeling and Forecasting 6 Subspace Characteristics of LMP Data 6.1 Introduction 6.2 Model and Distribution of LMP 6.3 Methodology 6.3.1 Problem Formulation 6.3.2 Basic Framework 6.3.3 Principal Component Analysis 6.3.4 Recursive Basis Search (Bottom-Up) 6.3.5 Hyperplane Detection (Top-down) 6.3.6 Short Summary 6.4 Case Study 6.4.1 Case 1: IEEE 30-Bus System 6.4.2 Case 2: IEEE 118-Bus System
6.4.3 Case 3: Illinois 200-Bus System 6.4.4 Case 4: Southwest Power Pool (SPP) 6.4.5 Time Consumption 6.5 Discussion and Conclusion 6.5.1 Discussion on Potential Applications 6.5.2 Conclusion References 7 Day-Ahead Electricity Price Forecasting 7.1 Introduction 7.2 Problem Formulation 7.2.1 Decomposition of LMP 7.2.2 Short-Term Forecast for Each Component 7.2.3 Summation and Stacking of Individual Forecasts 7.3 Methodology 7.3.1 Framework 7.3.2 Feature Engineering 7.3.3 Regression Model Selection and Parameter Tuning 7.3.4 Model Stacking with Robust Regression 7.3.5 Metrics 7.4 Case Study 7.4.1 Model Selection Results 7.4.2 Componential Results 7.4.3 Stacking Results (Overall Improvements) 7.4.4 Error Distribution Analysis 7.5 Conclusion References 8 Economic Impact of Price Forecasting Error 8.1 Introduction 8.2 General Bidding Models 8.2.1 Deterministic Bidding Model 8.2.2 Stochastic Bidding Model 8.3 Methodology and Framework 8.3.1 Forecasting Error Modeling 8.3.2 Multiparametric Linear Programming (MPLP)Theory 8.3.3 Error Impact Formulation 8.3.4 Overall Framework 8.4 Case Study 8.4.1 Measurement of STPF Error Level 8.4.2 Case1: LSE with Demand Response Programs 8.4.3 Case 2: LSE with ESS 8.4.4 Case3: Stochastic LSE Bidding Model 8.4.5 TimeConsumption 8.5 Conclusions and FutureWork References 9 LMP Forecasting and FTR Speculation 9.1 Introduction 9.2 Stochastic OptimizationModel 9.2.1 Model of FTR Portfolio Construction Problem 9.2.2 Scenario-Based Stochastic Optimization Model 9.3 Data-Dnven Framework
9.4 Methodology 9.4.1 Clustering 9.4.2 Mid-Term Probabilistic Forecasting 9.4.3 Copulas for Dependence Modeling 9.4.4 Training and Evaluation Timeline 9.4.5 Scenario Generation 9.5 Case Study 9.5.1 Data Description 9.5.2 Comparison Methods 9.5.3 Statistical Validation of Quantile Regression 9.5.4 Scenario Quality Evaluation 9.5.5 Impact of Node Reduction with Clustering 9.5.6 Revenue and Risk Estimation 9.5.7 Sensitivity Analysis on the Number of Clusters 9.6 Conclusion References Part Ⅲ Market Bidding Behavior Analysis 10 Pattern Extraction for Bidding Behaviors 10.1 Introduction 10.2 Assumptions and Proposed Framework 10.2.1 Model Assumptions 10.2.2 Bidding Data Format 10.2.3 Data-Driven Analysis Framework 10.3 Data Standardization Processing 10.3.1 Filtering Available Capacities 10.3.2 Sampling Bidding Curves 10.3.3 Unifying Data Length 10.3.4 Clipping Extreme Prices 10.4 Adaptive Clustering of Bidding Behaviors 10.4.1 Distance Measurement 10.4.2 K-Medoids Clustering 10.4.3 Adaptive Clustering Procedure 10.4.4 Clustering Algorithm 10.5 AEM Data Description 10.5.1 Description of Market Participants 10.5.2 Description of Bidding Data 10.6 Bidding Pattern Analysis 10.6.1 Parameter Setting 10.6.2 Bidding Patterns of DUs by Fuel Type 10.6.3 Comparison of Similar DUs 10.6.4 Discussion 10.7 Feature Analysis on Bids 10.7.1 Discrete Aggregation Feature 10.7.2 Probability Distribution Feature 10.7.3 Time Distribution Feature 10.8 Conclusions References 11 Aggregated Supply Curves Forecasting 11.1 Introduction 11.2 Market and Framework
11.2.1 Market Descriptions 11.2.2 Forecasting Framework 11.3 Data Integration and Feature Extraction 11.3.1 Data Integration 11.3.2 Feature Extraction 11.4 ASC Forecasting 11.4.1 LSTM Model 11.4.2 Influencing Factors 11.4.3 Training and Forecasting 11.4.4 Evaluation Criteria 11.5 Case Study 11.5.1 Dataset Description 11.5.2 Feature Extraction 11.5.3 ASC Forecasting 11.5.4 Calculation Information 11.5.5 Methods Comparison 11.6 Conclusion References 12 Learning Individual Offering Strategy 12.1 Introduction 12.2 Data-Driven Market Simulation Framework 12.2.1 Market Assumptions 12.2.2 Offering Data Clustering and Indexing 12.3 Individual Offering Strategy Learning 12.3.1 MFNN Model Structure 12.3.2 MFNN Model Inputs 12.3.3 MFNN Model Training 12.3.4 DNN-Based Model Structure 12.4 Market Clearing Simulation 12.5 Case Study 12.5.1 Basic Data 12.5.2 Individual Offering Behavior Forecasting 12.5.3 Market Simulation 12.5.4 Comparison with Current Price Forecasting Methods 12.5.5 Calculation Efficiency 12.6 Conclusions References 13 Reward Function Identification of GENCOs 13.1 Introduction 13.2 Assumptions and Framework 13.2.1 Market Assumptions 13.2.2 Data-Driven Framework 13.3 Bidding Decision Process Formulation 13.3.1 Markov Decision Process in Wholesale Markets 13.3.2 Reinforcement Learning Process 13.3.3 Bidding Data Integration 13.4 Reward Function Identification 13.4.1 Deep Inverse Reinforcement Learning Algorithm 13.4.2 Discretization Methods for States and Actions 13.5 Bidding Behavior Simulation
13.5.1 DQN-Based Bidding Simulation Model 13.5.2 Value Function and Q-Network 13.6 Case Study 13.6.1 Dataset Description 13.6.2 Parameter Setting 13.6.3 Reward Function Identification 13.6.4 Bidding Behavior Simulation 13.7 Conclusions References
[