 Original research
 Open access
 Published:
A trilevel programmingbased frequency regulation market equilibrium under cyber attacks
Protection and Control of Modern Power Systems volume 8, Article number: 55 (2023)
Abstract
Owing to their flexibility and rapid response, gridconnected distributed energy resources (DERs) are wielding growing influence in frequency regulation markets (FRMs). Nevertheless, compared with conventional largescale generators, smallscale DERs are usually weakly shielded by cyber security measures. This offers attackers the opportunity of disrupting the clearing and settlement of FRMs by manipulating the bid information of DERs. In this paper, the frequency regulation market equilibrium is studied considering the dynamic gaming between attackers and defenders, both of which need the knowledge of FRMs for their interventions. Specifically, a trilevel programming model characterizing the attacker–defender–operator (ADO) interdiction problem in FRMs is developed and then analyzed using a column and constraint generation algorithm, thereby achieving market equilibrium representing the defender's best response to the attacker. The defense strategy in the market equilibrium can attenuate the negative influence of cyber attacks upon the FRMs to the maximum extent. Finally, based on the operating rules of the California Independent System Operator, the FRM process considering the ADO interdiction is simulated and the numerical equilibrium results are presented.
1 Introduction
Distributed energy resources (DERs) such as smallscale wind and solar power were once deemed exogenous disturbances because of their intermittent generation. Nevertheless, when integrating with microgrids or virtual power plants (VPPs), DERs gradually become frequency regulation service providers by collectively offering their respective capacities. Energy aggregation attenuates the fluctuation of individual DERs via a smoothing effect, thereby assisting DERs to provide controllable reserves similar to largescale thermal and hydropower plants. Energy aggregation also enhances the bargaining power of DERs, transforming DERs from price takers to price makers. Although DERsbased frequency regulation has benefits such as reduced regulation costs and improved performance owing to the economy and quick response of DERs, the security risks of DERs' participation in frequency regulation markets (FRMs) cannot be ignored. The bottomlevel DERs are susceptible to cyber attacks because of their comparatively weak security measures. Attack propagation may finally collapse the energy aggregators designated as vulnerable FRM participants. Attackers can manipulate the bid information of vulnerable participants, thereby disrupting FRM processes for the attackers' benefit.
Since ancillary services play essential roles in reliable energy provision on a realtime basis, FRMs usually have more stringent requirements for FRM quality, e.g., all participating resources should demonstrate the ability to meet the control, telemetry, and minimum capacity requirements [1]. To reach the minimum threshold, spatially segregated DERs can integrate with VPPs to elevate collective capacity [2], and aggregators of VPPs then participate in FRMs on behalf of DERs. DERs in the same locality can form microgrids, and the corresponding aggregators will represent DERs to participate in FRMs [3]. As a result, researchers study the bidding behaviors of aggregators extensively. As for the battery energy storage (BES) aggregator, studies have proposed optimal bidding policies based on different market settings and BES characteristics. Although BES aggregators mainly target the provision of actual regulation by behaving as price takers [4], new BES aggregators begin to secure more operational profits by actively submitting bid prices [5]. Considering battery lifespan degradation, some studies also incorporate battery aging cost into the bidding strategies [6]. There is also research investigating realtime control policy to optimize the battery regulation response [7], whereas in [8], a BES optimal operation for both frequency regulation and energy arbitrage is considered and the optimization methodology for sizing and operating BES in distribution networks is developed. As well as pure BES, the bid of combined renewable energy and BES also attracts the attention of researchers in [9,10,11,12,13]. In [14], they investigate an aggregator controlling a fleet of electric vehicles (EVs) and energy storage (ES), and determine the optimal size of the aggregator's bids. By considering the uncertain energy and frequency regulation prices, the conditional valueatrisk method is employed to model the FRM risk, and a stochastic mixed integer linear programming model is established to obtain the optimal bids.
In contrast to aggregators, FRM operators aim to minimize the regulation cost on the condition that participants (including the aggregators) satisfy all market rules. Taking the American FRMs rules as an example, prior to FERC Order 755, cleared participants received no compensation for performing regulation. Fastramping aggregators tend to bear a more gratuitous burden of regulation than conventional units, thus discouraging quality resources from contributing to the regulation process. Therefore, Order 755 requires fair compensation for the frequency regulation service [15]. In recent years, various markets also propose different clearing strategies considering new FRM rules. In [16], researchers design a performance ratio that quantifies the relative effectiveness of generators and flexible demand resources in frequency regulation. Clearing processes considering response rates are studied. Reference [17] proposes an integrated dynamic market mechanism that combines the realtime and frequency regulation markets. Market players including renewable generators and flexible consumers can negotiate electricity prices using the most recent information on the available wind power and quality of grid frequency.
Instead of exploring new bidding or clearing strategies, this paper investigates how possible cyber attacks affect FRM processes including the clearing and the settlement results. It is known that the degree of security and operational reliability is usually positively correlated with the size of resources. The smaller the size of resources is, the lower the protection level is, and vice versa. DERs are usually of small capacity and equipped with comparatively weaker protection measures, and hence they are more likely to suffer cyber attacks. It is difficult for attackers to compromise strongly protected largescale generators like thermal or hydro units. Attackers may infiltrate DERs via various vulnerability points like unauthorized access to DER controllers and penetration through the facility network [18]. Through attack propagation in the aggregator network, the upperlevel aggregators may also suffer cyber security threats. Attackers can manipulate the bid information of these vulnerable aggregators to disrupt the market processes, i.e., the clearing and settlement.
Attackers can also exploit financial arbitrage opportunities through cyber attacks. Reference [19] analyzes how attackers can manipulate locational marginal prices (LMPs) in the electricity market by injecting malicious data into critical parameters. Unlike many attack models that neglect the characteristics of LMPs, reference [20] develops a new cyber attack strategy that not only bypasses bad data detection during state estimation but also conceals the compromised LMPs as normal LMPs to evade detection by market operators. Besides electricity markets, researchers have considered cyber attacks against various applications. Reference [21] proposes a load redistribution attack model using DC state estimation in which attackers have limited access to network topology and information. In [22], a new attack model using AC state estimation is studied. This takes into account both the cost of the attacker and the attack influence. As for the detection, reference [23] proposes a novel datadriven FDIA framework and designs an unsupervised detection scheme to detect the stealthy FDIA, whereas [24] constructs a novel method that employs graph theory principles for identifying false data injection attacks (FDIAs) on AC state estimation.
An attacker usually has no physical asset (the generating units), and hence, he or she chooses to collude with proxy participants and action so that the proxy participants can increase their compensation in the settlement. In this situation, the proxy participants are the stakeholders on behalf of the attacker. This profitoriented attack strategy will fundamentally change the settlement results, diverging the compensation from the expected value for other nonproxy participants. To reverse this undesired compensation deviation and maintain the compensation at a reasonable level, the defender could alter specific market parameters, e.g., the bidding data of the defensive unit. In summary, both the attacker and defender exert influence on the FRM (clearing) model to reach their specific attack and defense objectives. In return, the attacker and the defender form an interdependency relationship by exchanging information during the clearing process. In this paper, a trilevel programming model is used to characterize the aforementioned interaction among the defender, the attacker, and the FRM. The upperlevel attacker's model formulates the optimal compensation for the stakeholders. The middlelevel defender's model formulates the minimal compensation deviation, and the lowerlevel market model formulates the minimal regulation cost in the clearing process. The attacker and the defender form a noncooperative game relationship in the trilevel hierarchy, and the independent system operator desires to obtain a market equilibrium where each player selects the best response to the opponent's strategies. By linearizing the threelevel model and using the column and constraint generation (C&CG) algorithm, the FRM equilibrium is assessed under the attackeranddefender game.
The main contributions of this paper are:

Considering the vulnerabilities of DER aggregators that participate in FRMs along with conventional units, a new cyber attack scenario that targets the FRM is researched and the FRM equilibrium in attackeranddefender game scenarios is studied. The proposed FRM equilibrium evaluation mechanism can offer better insights and quantitative information on how the FRM clearing results evolve during advanced attackeranddefender interplay. Also, the proposed strategy has great applicability in the equilibrium evaluation of other marketoriented cyber attacks.

By formulating a novel attackeranddefender Stackelberg game, where both players interact and impact each other's decisions in the FRM, it analyzes the effect of having a profitdriven attacker and a balancefocused defender simultaneously operating in the FRM.

By linearizing the attackeranddefender Stackelberg game model and reformulating it into an equivalent bilevel optimization model, the C&CG algorithm is used to solve the bottomlevel subproblem and toplevel master problem in an iterative manner, thus achieving market equilibria more efficiently.
The remainder of the paper is organized as follows: Sect. 2 gives the basis of profitoriented cyber attacks in FRMs, while Sect. 3 presents the mechanism of FRMoriented cyber attacks via vulnerable aggregators. Section 4 describes the strategy of calculating the FRM equilibrium in a noncooperative game between the defender and the attacker. Section 5 gives the case studies and Sect. 6 draws the conclusions.
2 Profitoriented cyber attack in frequency regulation market
This section presents a theoretical background of profitoriented cyber attacks in FRMs. Specifically, it explains in detail the feasibility of compromising the energy aggregator and illustrates the complete attack process of obtaining lucrative compensation for the attacker.
2.1 Cyber security threats of distributed energy resources and energy aggregators
The largescale integration of DERs transforms the utilitycentric structure into a multiaggregator network. Inspired by the architecture in [18], a DER architecture is categorized into four levels as shown in Fig. 1.
Level 1 represents a collection of individual DER devices and the corresponding controllers, while level 2 mainly contains utility communication systems exchanging control commands and data with DER devices. Level 3 is the collection of aggregators integrated with multiple DERs, where aggregators represent DERs to participate in the FRM and other markets. Level 4 is the ISO supervising markets and operation of power systems. Such DER architecture is susceptible to cyber attacks. For instance, attackers may infiltrate the DER network in level 1 by exploiting protocol bugs. The attack can then propagate in the architecture if there is insufficient deployment of firewalls, security gateways, and other measures. The attacker eventually collapses the aggregator in level 3, which is entirely subject to the whim of the attacker when bidding in the FRM in level 4.
2.2 Profitmaking attack measures in the frequency regulation market
For brevity, those aggregators that suffer the cyber security threats in Fig. 1 are designated as vulnerable aggregators. When corrupted by attackers, vulnerable aggregators will submit compromised offer prices, thus disrupting the clearing and settlement results for the benefit of stakeholders, who represent attackers, in securing payments for providing frequency regulation services. A simple merit order model is used to show that attackers can enhance payments by elevating the offer prices of vulnerable units. Figure 2 shows the clearing process of a simplified FRM.
As seen, the market operator ranks four market participants based on ascending offer prices, and the clearing price is equal to the offer price of the marginal participant, i.e., the most expensive participant that is required to meet the inelastic reserve demand. When the offer price of the vulnerable aggregator (participant 1) is manipulated from to \(O_{1,co}^{{\text{c}}}\), a clearing price difference \(\Delta p_{{\text{cl}}}^{{\text{c}}}\) between the intact and compromised clearing prices occurs. The positive \(\Delta p_{{\text{cl}}}^{{\text{c}}}\) will increase the compensation for the stakeholder (participant 2) by \(\Delta p_{{\text{cl}}}^{{\text{c}}} r_{2}^{{\text{c}}}\). Obviously, price manipulation of the vulnerable aggregator will disrupt the original market equilibrium by changing the order of clearing and the expected compensation. Specifically, the expected compensation is studied in this paper. This refers to the expected regulation capacity payment the cleared participant can obtain. Although the attacker targets compensation increase of stakeholders, it also causes deviations in the expected compensation for other participants. In response, defenders try to restore equilibrium by reducing compensation deviations. In this paper, it assumes that the defender uses attack vs defense drills to evaluate the market equilibrium in the most severely adversarial situation where the attacker and defender form a noncooperative game relationship. It occurs when the attacker turns to an insider lurking in the DER architecture with complete information. The attacker desires to maximize the stakeholder's compensation (by manipulating vulnerable aggregators), while the defender hopes to minimize the expected compensation deviation (by dispatching defensive participants). Since both implementations are dependent upon a market clearing and interplay reciprocally, the defender should reassess the market equilibrium where both players achieve the best responses to their opponent.
3 Frequency regulation market equilibrium considering cyber attacks against vulnerable aggregators
From Sect. 2, we know the three basic truths about FRMs with aggregators. First, the attacker can infiltrate the DER architecture and compromise vulnerable aggregators. Second, the attacker can manipulate compromised vulnerable aggregators for its own benefit while causing expected compensation deviations. Third, the attacker and defender form a noncooperative game by interacting through the common market clearing process. The attacker moves before the defender, while the defender moves before the operator, leading to an ADO interdiction problem. Figure 3 gives the general framework of the noncooperative interplay between the attacker and the defender in the FRM.
In this section, the following two issues are further elucidated:

Question 1: How to formulate the ADO interdiction problem in the FRM?

Question 2: How to establish market equilibrium considering the noncooperative interplay between the defender and the attacker?
3.1 Trilevel programmingbased attacker–defender–operator interdiction
Here, a trilevel programming model is used to formulate the attacker–defender–operator interdiction problem. As profitoriented entities, attackers aim to maximize the capacity compensation of the stakeholders who represent attackers to trade in FRMs. Instead of minimizing attackers’ compensation, the primary goal of defenders is to minimize the attack influence on the originally desired market equilibria. The upperlevel model characterizes the attacker's objective of compensation maximization for the stakeholder, the middlelevel model characterizes the defender's objective of minimization of expected compensation deviations, and the lowerlevel model describes the market clearing process.
As the game leader, the attacker moves first by manipulating the offer prices of vulnerable aggregators. Then the defender evaluates the defense's goal under the given attack strategies. Finally, under the given attack and defensive strategies, the operator model completes the clearing process which passes the clearing results to the defender and the attacker to evaluate their respective objectives. Figure 4 demonstrates the above trilevel game hierarchy. In the following, the trilevel model is expanded from the lower to the upper levels.
3.2 Attacker model: increase of capacity compensation payment for the stakeholder
As mentioned in Sect. 2, the attacker manipulates the offer prices of vulnerable aggregators to increase the capacity compensation payment for the stakeholder. Specifically, the payment maximization model is used to characterize the attack problem, as:
where \(\mathcal{N}_{v}\) represents the set of vulnerable aggregators. The offer prices of vulnerable aggregators should be within certain limits, i.e.:
(1) Defender model: reduction of expected capacity compensation payment deviations: As mentioned in Sect. 2, the manipulation of offer prices of vulnerable aggregators will cause expected capacity compensation payment deviations. These deviations are detrimental to participants' interests. The twopart payment mechanism offers participants a marketbased capacity payment and a performance payment. In this paper, it supposes that the attacker and the defender only game over the capacity payment, and the defender wants to minimize the expected capacity payment deviations:
where \(j \in \mathcal{N}_{{\text{d}}}\) represents the set of defensive participants, and \(j \in \mathcal{N}_{{\text{n}}}\) represents the set of units other than defensive and vulnerable units. \(p_{{\text{cl}},0}^{{\text{C}}} r_{j,0}^{{\text{C}}}\) represents the expected capacity compensation payment for participant j. It assumes that the FRM is less competitive and reserve requirements are inflexible. In this situation, participants tend to submit the same bid. Hence, the defender can use the clearing results from the prior interval to obtain \(p_{{\text{cl}},0}^{{\text{C}}} r_{j,0}^{{\text{C}}}\). The defender uses offer prices of defensive participants to rectify the expected capacity compensation payment, and these offer prices should be within certain limits, as:
where \(O_{j,\min }^{{\text{C}}}\) and \(O_{j,\max }^{{\text{C}}}\) represent the minimum and maximum regulation capacity offer prices, respectively. \(O_{j,\min }^{{\text{M}}}\) and \(O_{j,\max }^{{\text{M}}}\) represent the minimum and maximum regulation mileage offer prices, respectively.
(2) Operator model: performancebased frequency control ancillary service market clearing formulation: FERC Order No. 755 requires ISOs or RTOs to offer FRM participants a twopart payment including a marketbased capacity payment and marketbased payment for performance [15]. Correspondingly, participants submit both the regulation capacity offer price and regulation mileage offer price. The resulting regulation cost minimizationoriented clearing model is thus written by:
where the subscript j represents the index for market participants. \(\mathcal{N}_{{\text{p}}}\) is the set of participants, while \(O_{j}^{{\text{C}}}\) and \(O_{j}^{{\text{M}}}\) represent the regulation capacity offer price and the regulation mileage offer price, respectively. \(r_{j}^{{\text{C}}}\) and \(r_{j}^{{\text{M}}}\) represent the cleared regulation capacity and regulation mileage, respectively.
The constraints of the clearing model are described in (8)–(13) below.
where R^{C} represents the regulation capacity requirement.
where m_{s} and m_{j} represent the systemlevel and the participantlevel mileage multiplier, respectively. \(R_{0}^{{\text{M}}}\) represents the regulation mileage requirement from the prior regulation interval, and U_{j} is the bid (maximum regulation capacity) of participant j. Equation (9) avoids mileage scarcity and situations where regulation mileage requirements may drive increased regulation capacity procurement above the regulation capacity requirement.
where U_{j} represents the bid for regulation capacity. Equations (10) and (11) present the operational limits.
According to FERC Order No. 755, cleared regulation mileage \(r_{j}^{{\text{M}}}\) of participant j should be no less than cleared regulation capacity \(r_{j}^{{\text{C}}}\), but no more than the product of its mileage multiplier m_{j} and the cleared regulation capacity. It follows that:
Based on (7)–(13), the trilevel ADO interdiction model can be rewritten as:
4 Market equilibrium in a noncooperative game between the defender and the attacker
The proposed trilevel ADO model is used to assess the market equilibrium where both the defender and the attacker achieve the optimal condition upon the strategy of their opponent. The study of market equilibrium under cyber attacks is essential for analyzing how the attacker and the defender may affect the market. This helps system operators make informed decisions about market intervention, trading correction, and even termination. Market equilibrium under an attackanddefense game can give system operators insights into how advanced attacker and defender interplay affects the market operation, thus guiding the postgame decisionmaking.
In this paper, Bender's primal decomposition framework is adopted to analyze the trilevel ADO model. The first step is to merge the middlelevel and lowerlevel problems into a singlelevel problem using either the strong duality theorem [25,26,27] or KKT optimality conditions [28]. Previous research mainly assumes that the defender takes proactive measures. In this case, the middlelevel and lowerlevel problems constitute a max–min bilevel subproblem. Instead, it assumes that the attacker moves before the defender in the Stackelberg game. This is because in many realworld situations, the attacker has the advantage of surprise and can launch an attack before the defender has a chance to react. From the third party's perspective, we can reformulate the trilevel model as a twostage optimization problem:
where \({\varvec{y}} \in {\varvec{S}}_{y}\) represents the feasible domain of the upperlevel variables, i.e., \(O_{j}^{{\text{C}}} ,O_{j}^{{\text{M}}} ,\forall j \in \mathcal{N}_{{\text{v}}}\). \({\varvec{x}} \in {\varvec{S}}_{x}\) represents the feasible domain of the middlelevel and lowerlevel variables, i.e., (5)–(6), and (7)–(13). The firststage problem corresponds to the upperlevel attacker problem, while the secondstage problem is to model the decisionmaking of the FRM operator after the attack and defense strategies are revealed.
It is noted that the secondstage problem is a bilevel problem. Hence singlelevel reduction is performed using the Karush–Kuhn–Tucker (KKT) conditions. The Lagrangian function of the lowerlevel optimization model is given as:
where r is the set of cleared capacity and mileage, and λ is the set of dual variables. \(\lambda_{j}^{{\text{C}}\min}\) and \(\lambda_{j}^{{\text{C}}\max}\) are the dual variables corresponding to (10) and (11), respectively, while \(\lambda_{j}^{{\text{M}}\min}\) and \(\lambda_{j}^{{\text{M}}\max}\) are the dual variables corresponding to (12) and (13), respectively. Then, KKT optimality conditions are (8), (9) and:
The subproblem model is expressed as:
where \(\Xi = \{ {\varvec{r}},p_{{\text{cl}}}^{{\text{C}}} ,p_{{\text{cl}}}^{{\text{M}}} ,{\varvec{\lambda}}\}\). The constraints are (5), (6), and (17)–(20). It is noted that both the objective function and (19)–(22) have nonlinearity. As for \(p_{cl}^{{\text{C}}} r_{j}^{{\text{C}}}\) in the objective function, the McCormick Envelope relaxation is used to transform it into a linear term, making the new objective a quadratic one. The complementary slackness in (17)–(32) is linearized using big M methods. Finally, the subproblem is transformed into a mixed integer quadratic programming problem, as:
Appendix gives the details of (24). The master problem is formulated as a mixed integer quadratically constrained programming problem, as:
Appendix also gives the details of (25). Considering the integer variables in the subproblem, Bender's dualcutting plane algorithm cannot be used to gradually construct the value function of the master problem using dual solutions of the subproblem. Therefore, the columnandconstraint generation (C&CG) method is used to obtain the market equilibrium based on (24) and (25). This dynamically generates constraints with subproblem decision variables in the primal space [29]. The complete C&CGbased market equilibrium computational procedure is as follows:

Step 1: Set the lower bound \(LB =  \infty\) and upper bound \(UB =  \infty\). Set the initial attack strategies \(O_{j}^{{\text{C}}} = O_{j,0}^{{\text{C}}} ,O_{j}^{{\text{M}}} = O_{j,0}^{{\text{M}}}\), and \(\forall j \in \mathcal{N}_{{\text{v}}}\), and then send them to the subproblem.

Step 2: Solve the subproblem and obtain the value of the objective function. Update the upper bound as \(UB = \min (UB,\sum\nolimits_{{j \in N_{{\text{n}}} }} {(\omega_{j}^{{\text{C}}} }  p_{{\text{cl}},0}^{{\text{C}}} r_{j,0}^{{\text{C}}} )^{2} )\). Obtain the defensive strategies \(O_{j}^{{\text{C}}} ,O_{j}^{{\text{M}}}\), and \(\forall j \in \mathcal{N}_{{\text{d}}}\), and send them to the master problem.

Step 3: Solve the master problem and obtain the value of the objective function. Update the lower bound as \(LB = \max (LB,\sum\nolimits_{{j \in N_{{\text{a}}} }} {\omega_{j}^{{\text{C}}} } + \eta )\). If \((UB  LB)/LB \le CT\), stop; otherwise, obtain the attack strategies \(O_{j}^{{\text{C}}} ,O_{j}^{{\text{M}}}\), and \(\forall j \in \mathcal{N}_{{\text{v}}}\), send them to the subproblem, and then go to Step 2.
5 Simulations and numerical analyses
In this section, the market equilibrium of an FRM guided by CAISO rules is evaluated. The FRM contains 15 participants with No. 12 the defensive participant and No. 13–15 the vulnerable participants. Table
1 gives the corresponding FRM information. The systemlevel mileage multiplier is 3.26, and the regulation capacity requirement R_{C} is 1000 MW.
Figure 5 shows the intact FRM clearing results without considering the defender–attacker interaction. As can be seen, participants No. 6, 10, and 12 have empty bars, meaning that they fail to secure the bid. This is because they submit either high regulation capacity offer prices or regulation mileage offer prices, so none of them secures the bid after the FRM is cleared. Figure 6 gives the results of capacity payments for all cleared participants. The stakeholder on behalf of the attacker can get $3980. When only the presence of the attacker is considered, the attacker and the FRM form a bilevel game, and the clearing results are shown in Fig. 7. As can be seen, in order to increase the capacity payment for the stakeholder by increasing the cleared price, the attacker manipulates the offer price of vulnerable units (No. 13, 14, and 15) to high values. Hence, vulnerable participants No. 13 and 14 fail to secure the bids after attack. In this situation, the marginal units No. 5 and 15 compensate for the imbalance of capacity and mileage because of the phaseout of No. 13 and 14, while the cleared capacity and mileage of the remaining units remain the same.
Figure 8 gives the results of capacity payments for all cleared participants. As can be seen, with the presence of the attacker and the defender, the stakeholder on behalf of the attack can get $10,932, which is far larger than the previous $3980, meaning that the attacker can greatly increase its compensation in the bilevel game. Figure 9 shows that the attacker would disrupt the market equilibrium. Participants No. 1–4, 7–8, and 11 experience 174.7% of payment increase, and participant No. 5 experiences 740.7% of increase. The elevation of offer prices of the vulnerable No. 13–15 not only increase the compensation for the stakeholder but also other units, causing financial losses to the operator. Since participants 6 and 10 do not win any bids either in the nonattack or the attack situations, there is no compensation for them. Hence, participants 6 and 10 have no payment in both situations.
To restore market equilibrium, the defender uses defensive unit No. 12 to change its offer price and correct the market clearing process. As can be seen, the clearing results for the participants that are neither vulnerable aggregators nor defensive units in Fig. 10 are approximately the same as those in the intact FRM shown in Fig. 5. Figure 11 shows that the capacity payment for the stakeholder decreases from the original $10,932 to $4938, which means the defense can negatively affect the gain the stakeholder makes on behalf of the attacker. Figure 12 shows the capacity payment deviations for the participants. As can be seen, participants No. 1–4, 7–8, and 11 experience 24.1% of deviation, which is far smaller than the 174.7% in Fig. 9. Participant No. 5 experiences 119.6% of deviation, which again is far smaller than the 740.7% in Fig. 9. It means that the defender can rebalance the FRM equilibrium in the attackanddefense game.
6 Conclusions
In this paper, a novel profitoriented cyber attack in the FRM is studied and the noncooperative game relationship between the attacker and the defender is analyzed. By formulating a trilevel game considering the respective goals of the attacker and the defender, the FRM equilibrium in the attackanddefense game situation is assessed. The case studies show that the participation of the defender can significantly rebalance the FRM equilibrium.
Availability of data and materials
The datasets used and/or analyzed during the current study are available in Sect. 5 Simulations and Numerical Analyses.
References
Helman. U. (2021). Demand response in the us wholesale markets: Recent trends, new models, and forecasts. In Variable generation, flexible demand (2nd ed, ch 10, pp. 211–257). Academic Press.
Chen, W., Qiu, J., Zhao, J., et al. (2021). Bargaining gamebased profit allocation of virtual power plant in frequency regulation market considering battery cycle life. IEEE Transactions on Smart Grid, 12(4), 2913–2928.
Srivastava, P., Chang, C. Y., & Cortés, J. (2022). Enabling der participation in frequency regulation markets. IEEE Transactions on Control Systems Technology, 30(6), 2391–2405.
Arteaga, J., & Zareipour, H. (2019). A pricemaker/pricetaker model for the operation of battery storage systems in electricity markets. IEEE Transactions on Smart Grid, 10(6), 6912–6920.
Dong, Y., Dong, Z., Zhao, T., et al. (2021). A strategic dayahead bidding strategy and operation for battery energy storage system by reinforcement learning. Electric Power Systems Research, 196, 66.
Wang, Y., Zhou, Z., Botterud, A., et al. (2016). Stochastic coordinated operation of wind and battery energy storage system considering battery degradation. Journal of Modern Power Systems and Clean Energy, 4(4), 581–592.
Xu, B., Shi, Y., Kirschen, D. S., et al. (2018). Optimal battery participation in frequency regulation markets. IEEE Transactions on Power Systems, 33(6), 6715–6725.
Wu, X., Zhao, J., & Conejo, A. J. (2022). Optimal battery sizing for frequency regulation and energy arbitrage. IEEE Transactions on Power Delivery, 37(3), 2016–2023.
He, G., Chen, Q., Kang, C., et al. (2017). Cooperation of wind power and battery storage to provide frequency regulation in power markets. IEEE Transactions on Power Systems, 32(5), 3559–3568.
GonzálezGarrido, A., SaezdeIbarra, A., Gaztanaga, H., et al. (2019). Annual optimized bidding and operation strategy in energy and secondary reserve markets for solar plants with storage systems. IEEE Transactions on Power Systems, 34(6), 5115–5124.
Fang, Y., & Zhao, S. (2020). Lookahead bidding strategy for concentrating solar power plants with wind farms. Energy, 203, 66.
Xie, Y., Guo, W., Wu, Q., et al. (2021). Robust mpcbased bidding strategy for wind storage systems in realtime energy and regulation markets. International Journal of Electrical Power & Energy Systems, 124, 66.
Yang, X., Fan, L., Li, X., et al. (2023). Dayahead and realtime market bidding and scheduling strategy for wind power participation based on shared energy storage. Electric Power Systems Research, 214, 66.
Vatandoust, B., Ahmadian, A., Golkar, M. A., et al. (2019). Riskaverse optimal bidding of electric vehicles and energy storage aggregator in dayahead frequency regulation market. IEEE Transactions on Power systems, 34(3), 2036–2047.
Chen, Y., Leonard, R., Keyser, M., et al. (2015). Development of performancebased twopart regulating reserve compensation on miso energy and ancillary service market. IEEE Transactions on Power Systems, 30(1), 142–155.
Yang, Y., Peng, J.C.H., & Ye, Z.S. (2021). A market clearing mechanism considering primary frequency response rate. IEEE Transactions on Power Systems, 36(6), 5952–5955.
Shiltz, D. J., Cvetković, M., & Annaswamy, A. M. (2016). An integrated dynamic market mechanism for realtime markets and frequency regulation. IEEE Transactions on Sustainable Energy, 7(2), 875–885.
Qi, J., Hahn, A., Lu, X., et al. (2016). Cybersecurity for distributed energy resources and smart inverters. IET CyberPhysical Systems: Theory Applications, 1(1), 28–39.
Xu, H., Lin, Y., Zhang, X., et al. (2020). Power system parameter attack for financial profits in electricity markets. IEEE Transactions on Smart Grid, 11(4), 3438–3446.
Zhang, Q., Li, F., Cui, H., et al. (2021). Marketlevel defense against FDIA and a new LMPdisguising attack strategy in realtime market operations. IEEE Transactions on Power Systems, 36(2), 1419–1431.
Khazaei, J. (2021). Cyberattacks with limited network information leading to transmission line overflow in cyber–physical power systems. Sustainable Energy, Grids and Networks, 27, 66.
Lu, K.D., & Wu, Z.G. (2022). Multiobjective false data injection attacks of cyber–physical power systems. IEEE Transactions on Circuits and Systems II: Express Briefs, 69(9), 3924–3928.
Chen, C., Wang, Y., Cui, M., et al. (2022). Datadriven detection of stealthy false data injection attack against power system state estimation. IEEE Transactions on Industrial Informatics, 18(12), 8467–8476.
Jorjani, M., Seifi, H., & Varjani, A. Y. (2021). A graph theorybased approach to detect false data injection attacks in power system ac state estimation. IEEE Transactions on Industrial Informatics, 17(4), 2465–2475.
Lai, K., Illindala, M., & Subramaniam, K. (2019). A trilevel optimization model to mitigate coordinated attacks on electric power systems in a cyberphysical environment. Applied Energy, 235, 204–218.
Wu, X., & Conejo, A. J. (2017). An efficient trilevel optimization model for electric grid defense planning. IEEE Transactions on Power Systems, 32(4), 2984–2994.
Davarikia, H., & Barati, M. (2018). A trilevel programming model for attackresilient control of power grids. Journal of Modern Power Systems and Clean Energy, 6(5), 918–929.
Ruiz, C., & Conejo, A. J. (2015). Robust transmission expansion planning. European Journal of Operational Research, 242(2), 390–401.
Zeng, B., & Zhao, L. (2013). Solving twostage robust optimization problems using a columnandconstraint generation method. Operations Research Letters, 41(5), 457–461.
Acknowledgements
Not applicable.
Funding
This work was supported by the National Natural Science Foundation of China (Grant No. 52207142) and Natural Science Foundation of Jiangsu Province (BK20210512) and Natural Science Foundation of the Higher Education Institutions of Jiangsu Province (20KJB510050).
Author information
Authors and Affiliations
Contributions
YW and CC contributed to methodology formulation, analysis, article drafting and writing; SZ contributed to the supervision; YL contributed to software; CH contributed to the visualization; YD contributed to the validation. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this article.
Appendix
Appendix
1.1 Mixed integer quadratic programming modelbased subproblem
The objective function is rewritten as:
where \(p_{{\text{cl}},\min}^{{\text{C}}}\) and \(p_{{\text{cl}},\max}^{{\text{C}}}\) represent the lower and upper limits of \(p_{{\text{cl}}}^{{\text{C}}}\), while \(r_{j,\min }^{{\text{C}}}\) and \(r_{j,\max }^{{\text{C}}}\) represent the lower and upper limits of \(r_{j}^{{\text{C}}}\).
The complementarity conditions in (19) to (22) can be rewritten as:
where M^{P} and \(M^{{\mu P}}\) are large enough constants, while \(\lambda_{j}^{{\text{C}}\min} ,\lambda_{j}^{{\text{C}}\max} ,\lambda_{j}^{{\text{M}}\min}\), and \(\lambda_{j}^{{\text{M}}\max}\) are binary variables.
The final subproblem model includes the objective (26), and constraints (8)–(9), (5)–(6), (17)–(18), and (27)–(42).
1.2 Mixed integer quadratically constrained programming modelbased master problem
As with the subproblem, the objective of the master problem is:
The constraints are (8)–(9), (2)–(3), (17)–(18), (31)–(42), and
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Wang, Y., Chen, C., Zhang, S. et al. A trilevel programmingbased frequency regulation market equilibrium under cyber attacks. Prot Control Mod Power Syst 8, 55 (2023). https://doi.org/10.1186/s41601023003328
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s41601023003328