Household Energy Standing Survey (hereafter “HESS”) is prepared by Korea Energy Economics Institute (KEEI, 2022), and this yearly survey collects various information, including household characteristics, appliance ownership, and energy consumption, including electricity. The previous surveys included 2,520 households, but HESS 2019 expanded the survey subjects to 8,010 households in 17 municipalities. The final survey result provided information of 6,597 surveyed households. This survey provides a wide range on information of surveyed households, including some information of usage patterns of the home appliances, household characteristics, such as income, number of household members, and age of household head, and housing features. The regional composition of the survey is quite similar to the actual regional composition of households in 2019, but there are limited numbers of single-person households in the survey. In 2020, the single-person household accounted for approximately 31% of the entire household, but only about 17% of the surveyed households were single-person households. The distribution of the monthly income of surveyed households is largely located between 200 million KRW and 600 million KRW. The shares of surveyed households with monthly income between 200 million KRW and 400 million KRW and between 400 million KRW and 600 million KRW are 33.4% and 30.7%, respectively. The survey provides information on appliance ownership and usage and consumption of various energy types, but this study focuses on the electricity consumption of households.

In this study, Support Vector Machine, Random Forest, and Decision Tree classifiers are used to classify the household’s electricity consumption in Korea. Support Vector Machine is a popular machine learning method in classification-type research questions. This is a supervised machine learning method, and it classifies the dataset by finding the optimal hyperplane separating data points of different classes. The hyperplane separates the data points of different classes, and the hyperplane with maximum margin, which is the distance between the hyperplane and the closest data point, is considered as the optimal one. The following equation is the objective of the Support Vector Machine classifier that maximizes the margin by minimizing ||w||² with a penalty of misclassified data points (Winters-Hilt and Merat, 2007)

Fig. 1. 
Graphical illustration of support vector machine

(Source: Winters-Hilt and Merat, 2007)

Fig. 2. 
Graphical illustration of random forest (source: Moon, 2022)

The study uses ‘Scikit-learn’ library to conduct the Support Vector Machine, and the kernel function is allowed to use. The kernel function transforms data to a higher-dimensional space and supports the classification of the dataset, which is difficult to be linearly separated. The study uses ‘polynomial’, ‘radial basis function’, ‘sigmoid’, and ‘linear’ kernels provided by the library.

This method considers the distance between the hysperplane and data points, so it is necessary to standardize the dataset (Hsu et al., 2016; Menon, 2009; Murty and Raghava, 2016; Park, 2019; Winters-Hilt and Merat, 2007). Particularly, this dataset contains various categorical and numerical variables, so this study applies ‘StandardScaler’. This standardization approach uses a z-score for standardization, and ‘OneHotEncoding’ from the scikit-learn library is used. A feature of this is to create a binary column for each category and assigns a value of one to the feature for categorical variables.

Next, the Random Forest classifier is an ensemble method establishing multiple decision trees with a subset of the dataset. Each tree makes its own prediction, and it aggregates the predictions, and selects the class with the highest votes. It splits the trees based on the Gini impurity, which is a probability of incorrect classification of randomly chosen data. The strengths of this method are limiting overfitting issues, handling large datasets with various features, and information of importance for feature selections, which help the understanding of the classification process.

The last method used in this study is decision tree classification. It is a supervised machine learning method, and it splits datasets based on the decision rule. The highest node is called the root node, where the split starts. It moves to the decision nodes with branches, and the leaf nodes are the final output of decisions (Géron, 2019; Park, 2019).

These methodologies are typically used for classification problems. While Support Vector Machine is considered as black box, Random Forest and Decision Tree suggest feature importance, which indicates the relative importance of a feature to other features. The comparison of three classification methodologies indicates their classification abilities based on various types of variables, and Random Forest and Decision Tree classification methodologies provide the relative importance of features in classification and suggest some interpretability of the classification results.

Fig. 3. 
Preprocessing and application of machine learning methodologies

The dataset includes various categorical and numerical variables. While it provides numerical household electricity consumption (kwh), the information on many variables is provided as categorical or dummy variables. Instead of directly using the information, the study set a classification research question, which focuses on how each variable affects the classification of the dependent variable. Thus, the study converted the dependent variable, the electricity consumption of households, from a numerical to a categorical variable. The study excluded some surveyed samples with missing key variables, such as floor level, and the final dataset for the analysis was 6,557 surveyed households.

Moreover, this study utilizes the household survey, and the survey contains various household and housing characteristics, which are fixed during the year. It is difficult to reflect the impacts of electricity prices and the seasonal differences. To indirectly consider those factors, the study considers the bracket of the progressive pricing structure in Korea in determining the classes of the dependent variable. Moreover, the residential electricity demand reaches a peak during summer (KEPCO, 2019), and the Korea Electric Power Corporation sets different ranges for the progressive pricing structure during summer, so the study considers summer separately with a different set of independent variables.

This study constructs a classification problem to understand the residential electricity consumption. A large portion of variables in survey information and used in this study is categorical variable, so there are limitations to use those variables to train the models and predict or forecast the amount of household electricity consumption. Instead, the study establishes a classification problem of a household’s electricity consumption with consideration of the progressive electricity pricing, which affect the electricity consuming behaviors of each household. The study converted the annual electricity consumption of each household into monthly average electricity consumption. Considering a number of categorical variables in the dataset, the study divided the household’s monthly average electricity consumption into four groups. To divide the groups, the study considered the progressive electricity pricing in Korea. However, a large number of households consume the level 2 pricing between 200 kWh and 400 kWh per month, so the study sets the range of electricity consumption as 100 kWh, instead of 200 kWh suggested by the progressive electricity pricing. Thus, the study sets four groups of monthly electricity consumption as following: (Group 1) below 200 kWh per month; (Group 2) 200 kWh ~ 300 kWh per month; (Group 3) 300 kWh ~ 400 kWh per month; and (Group 4) above 400 kWh per month. In addition, this study conducted a similar process to the household’s monthly electricity consumption during summer time (June – August). During summer, the electricity demand for cooling increases significantly, and KEPCO tends to ease the progressive billing during summer to reduce the economic burden of households. Thus, the study set four groups of monthly electricity consumption during summer time as follows: (Group 1) below 250 kWh per month; (Group 2) 250 kWh ~ 350 kWh per month; (Group 3) 350 kWh ~ 450 kWh per month; and (Group 4) above 450 kWh per month.

Also, the study selected key independent variables from various household and housing characteristics. The following table indicates the variables selected as the independent variables in this study. Based on the ownership and usage information of key appliances provided by the survey, the study calculates the monthly average consumption of electricity by the key appliances. The survey information provides information, such as type and size, with different units (e.g., Wh per use, Wh per kg, and W for washing machine), so the study calculates the average electricity consumption of each appliance by using appliance’s technical specification and usage information for securing the comparability. If there is only ownership information without usage, the study considers the number of appliances owned by the household. Also, the study created dummy variables of “single” and “old” to understand whether single-person households or households with members aged 65 or older affect the classification.

The different sets of variables are considered in understanding the household’s electricity consumption in two different time setting in this study. There are distinctive differences of using appliances in different seasons, including the use of heating and cooling devices. During the summer, the use of heating appliances would be quite limited, and the city gas and oil are mainly used for heating, so the study excludes heating-related variables in summer. The study sets two different specifications: one considers all characteristics, including ownership and usage of appliances, and the other specification considers variables closely related to the summer season and excluded characteristics, such as heating-related information.

The study applies three machine learning methodologies, which are Support Vector Machine, Random Forest, and Decision Tree, and conducts the classification of the household’s electricity consumption. Fig. 4 suggests the classification result of the Support Vector Machine. RandomizedSearchCV is used to find the optimal hyperparameters of the Support Vector Machine. Instead of checking all combinations of parameter values, this method randomly selects the parameter values from the specified distributions. While the performance of this method could be less well, it could reduce the computation burden and require a shorter time. The types and range of the hyperparameters in the search are suggested in the following table.

Fig. 4. 
Confusion matrix (support vector machine, annual)

The selected hyperparameters from RandomizedSearchCV are {‘C’: 100, ‘gamma’: 0.001, ‘kernel’: ‘rbf (radial basis function’’}, and the confusion matrix is as follows. As shown in the performance metrics, the weighted averaged f1-score is 0.54. By each class, the model predicts relatively well (65%) in class 2, and the classification performance of class 3 and class 4 are somewhat similar, but it provides relatively bad performance in classifying class 1 (34%). The result shows that the model predicts a large number of test samples as class 2 (200 kwh ~ 300 kwh per month), where a large number of observations are located. The classification result of the Support Vector Machine indicates the surveyed households’ characteristics and the ownership and usage of appliances are not very different to precisely classify the household’s monthly electricity consumption, particularly of class 1 with monthly consumption below 200 kwh per month.

Next, the study applies the Random Forest classifier to classify the same dataset and uses RandomizedSearchCV to find the optimal hyperparameters. The selected hyperparameters are {‘n_estimators’: 474, ‘min_samples_split’: 2, ‘min_samples_leaf’:5, ‘max_features’: ‘sqrt’, ‘max_depth’: 44, ‘bootstrap’: ‘false’, ‘random_state’:0}, and the confusion matrix and the performance metrics are as follows. The result is quite similar to the case of Support Vector Machine classification. A distinctive feature of Random Forest classification is the feature importance, which computes the level of the contribution of each feature in the model for decreasing the impurity. In this model, the highest feature is ‘size of housing’, and the usage information of appliances, such as tv, air conditioner, refrigerator, and cooker, are considered as the next important features. Some housing characteristics, such as main heating fuels, city, and cooling temperature, as well as household characteristics, such as household income and number of household members also showed relatively high feature importance.

Fig. 5. 
Confusion matrix (random forest, annual)

Lastly, the decision tree classifier is applied, and the range of hyperparameters are {‘max_depth’: list(range(3,10)), ‘min_samples_split’: [2,3,4]}. The selected hyperparameter is {‘max_depth’:5}, and the classification result and the feature importance are shown as follows. The performance metrics are relatively worse than the previous two models. Interestingly, this classification suggests ‘single’ as the highest feature importance, and ‘size of housing’ and ‘main heating fuels’ as the next important features. This result seems reasonable as the dependent variable is the classification result of a household’s monthly electricity consumption, and the single-member households would be likely to be classified as lower classes with less electricity consumption.

Fig. 6. 
Feature importance (random forest, annual)

Fig. 7. 
A part of decision tree (annual)

Fig. 8. 
Feature importance (decision tree)

Fig. 9. 
Confusion matrix (support vector machine, summer)

The results indicate that the random forest model shows slightly better performance compared to the other two methodologies. The feature importance of the random forest and decision tree models identify the housing characteristics, such as size of housing, the electricity consumption of appliances, and the number of household members as the important features in common. However, those models show different results in household’s characteristics. While random forest model finds ‘single’ as a relatively less important feature and ‘income’ as a relatively important feature, decision tree model identifies ‘single’ as the most important feature and does not find ‘income’ as an important feature.

In addition to the monthly average electricity consumption, the study classifies the household’s monthly electricity consumption in summer (June – August). For Support Vector Machine, the same methods and the range of hyperparameters are considered. The selected hyperparameters are {‘C’: 10, ‘gamma’: 0.1, ‘kernel’: ‘rbf (Radial Basis Function)’}. The confusion matrix and the performance metrics are as follows. Similar to the previous case, a large number of test samples are classified as class 2, but the f1-score of classifying class 1 becomes slightly higher than the previous case. Partly, this can be explained that the change in the range of monthly electricity consumption leads to more samples being classified as class 1 and enhance the classification performance. However, the general weighted average f1-score is 0.43, which is relatively lower than that of monthly electricity consumption for the entire year.

Next, The Random Forest classifier is applied to classify the household’s monthly electricity consumption during summer time. The selected hyperparameters are {'random_state': 0, 'n_estimators': 87, 'min_samples_split': 4, 'min_samples_leaf': 3, 'max_features': 'log2', 'max_depth': 50, 'bootstrap': False}. The confusion matrix and the performance metrics are as follows. The performance metric of the weighted average of the F1-score is slightly better than those of Support Vector Machine, but this result indicates relatively low performances in classifying large electricity-consuming households with relatively small samples. Similar to the annual case, the feature importance shows ‘size of housing’ as the highest feature, and the usage information of appliances are considered as the next important features. Also, the cooling temperature, number of windows, and province, also show some importance in classification.

Fig. 10. 
Confusion matrix (random forest, summer)

Fig. 11. 
Feature importance (random forest, summer)

For the decision tree classifier, the selected hyperparameter is {‘max_depth’:7, ‘min_samples_split’:4}, and the classification result and the feature importance are shown as follows. The performance metrics indicate similar results to the previous two models. Similar to the annual case, it finds ‘single’ feature as the highest importance and recognizes ‘size of housing’ as the next important feature. Also, ‘cooling temperature’ and ‘number of household members’ also show some importance in classification.

Fig. 12. 
A part of decision tree (summer)

Fig. 13. 
Feature importance (decision tree, summer)

Similar to the previous case, random forest model indicates slightly better performance compared to the other models. While the feature importance of random forest model shows a similar result to the previous case, it finds ‘number of windows’ as relatively more important but ‘main heating fuel’ as relatively less important. Also, decision tree classifier finds ‘cooling temperature’, ‘the electricity consumption of air conditioner’, and ‘income’ as important features. Compared to annual electricity consumption, features, such as ‘cooling temperature’ and ‘air conditioners’, which are more relevant to summer electricity consumption, show a relatively higher importance.