Machine Learning Based Operating Region Extension of Modular Multilevel Converters Under Unbalanced Grid Faults

The capacitor voltage ripples of the modular multilevel converter (MMC) are increased under unbalanced grid fault conditions. Since high capacitor voltage ripples deteriorate their lifetimes and may even cause tripping of the MMC system, it is important to restrict them. To this end, it is well known that injecting double fundamental frequency circulating currents can reduce the capacitor voltage ripples. However, finding a proper circulating current reference to achieve desired ripples analytically is complicated. This letter proposes an alternative method to quickly calculate the proper circulating current references without analytical computations, which is achieved by an artificial neural network (ANN) trained to approximate the relationship between circulating current references and capacitor voltage ripples. The training data are first extracted from a detailed simulation model of the MMC. Afterward, the ANN is trained by the input–output data to obtain the mapping relationship, which is then used to derive the desired circulating current references. Both the simulation and the experimental results verify the practicability of the proposed method, where the operating region can be extended 30% at a minimum in all testing conditions.

result, the capacitor voltage ripples will be unbalanced and will present higher amplitudes [2]. High SM capacitor voltages will cause tripping of the MMC system immediately if they exceed the safe voltage limit [3]. Therefore, the safe operating region of the MMC is limited during such faults [4]. Moreover, the SM capacitors will deteriorate more quickly through the aging process if they are subjected to high-voltage operation [5]. Aging gradually reduces their capacitance, and with aging the capacitor voltage will get even higher under unbalanced grid fault conditions. Several approaches have been proposed to address operation region extension by reducing SM capacitor voltage ripples under different grid conditions. In [4], the operating region is extended by injecting the second-order harmonic circulating current under single-phase grid fault condition. However, in this letter, the amplitude of injected ac circulating current is fixed, which means that different three-phase unbalanced grid fault conditions have not been addressed. Therefore, this method cannot flexibly control the voltage according to different fault conditions. In [6], the second-order harmonic circulating current injection method is combined with the optimized zero-sequence voltage to extend the operation region. Still, the method in [6] is only designed for balanced grid condition. In [7], ac circulating current reference is derived from the instantaneous capacitance with the upper and lower arm under single-phase grid. Then, the SM capacitor voltage ripple can be reduced. This instantaneous method is also applied under unbalanced grid condition in [8], and the results showed that this method can also reduce the SM capacitor voltage ripple under unbalanced grid conditions. However, these papers applied the method of injecting circulating currents regardless of whether SM voltages of these phases exceed the safe limit or not. If so, unnecessary power losses will occur in the phase that does not exceed the safe voltage limit. In [9]- [11], analytical equations are introduced to model SM voltage ripple under balanced grid condition. However, these ripple equations do not consider the influence of unbalanced grid condition, since deriving the equations for the unbalance-grid-condition case would be very complicated and time-consuming because of the extra negative-sequence current/voltage. Their derivation also requires strong control and domain expertise, which makes them difficult to reproduce and understand by practicing engineers. In [12]- [14], the components of the circulating current under unbalanced grid conditions are analyzed and a circulating current control is proposed in 0278-0046 © 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
order to suppress all the ac components of circulating current. However, suppressing all the ac circulating currents will not guarantee that the voltage ripples in the SM capacitors do not exceed the voltage limits. In [15], an analytical relationship between MMC arm power and circulating current references is derived. Then, the circulating current reference that reduces the SM capacitor voltage ripple is calculated based on derived equations. However, the work in [15] only analyzes the arm power of the MMC with different control techniques, whereas the accurate relation between SM capacitor voltage ripples and circulating current references is not discussed. In other words, the theoretical foundation set in [15] is not sufficient to explicitly derive circulating current references for controlling three-phase SM capacitor voltage ripples. In [3], an offset pulsewidth modulation and zero-sequence voltage injection method is proposed. The three-phase offsets added to the duty ratios to reduce the average SM capacitor voltages of the arms during unbalanced grid conditions [3]. However, this letter still does not address the accurate relation between SM capacitor voltage ripples and circulating current references. In [16], a nonlinear optimization method that can extend the operating region by the online model predictive method is proposed. Moreover, this method is combined with frequency-domain models of MMC, which can eliminate the need for time-consuming simulations. However, in this letter, the overall control structure is different from the widely use traditional MMC control structure because of the nonlinear H-infinity output current controller and predictive control regulator. This new control structure with nonlinear controller requires high knowledge level for implementation, which may make the practical application quite challenging. As abovementioned discussion underlies the need of developing simple and easily generalizable control methods for compensation of SM capacitor voltage ripples under unbalanced grid conditions. This letter proposes a flexible and computationally light ripple-compensation technique based on trained artificial neural network (ANN) that acts as a fast surrogate model mapping the relationship between circulating-current references and SM capacitor voltage ripples. With the help of this ANN model, the injected circulating current references are calculated automatically according to different grid conditions. In addition, desired capacitor-voltage-ripple values, which are limited to some predefined operating range, can be obtained explicitly. The ANN is trained by the data collected from simulations, and the application of the proposed method has been experimentally demonstrated.

A. Deployment of ANN
Several papers have used ANN to solve power electronics problems, from selecting weighting factors in model predictive control of power electronics converters [17] to automatically designing the power electronics system for reliability [18]. ANN is known to be a universal function approximator. In other words, it can in theory approximate any function with arbitrary precision, given that the number of layers and neurons has been appropriately chosen [19]. In practice, ANN structure is usually selected by trial-and-error.
The surrogate model in this letter is constructed by using a three-layer ANN, where there are nine neurons in the hidden layer. This surrogate model represents the relationship between circulating current, grid dip severity factor, and injected circulating current amplitude. In particular, this network represents the following relation: whereÎ cACa ,Î cACb , andÎ cACc are circulating currents,V cua , V cub , andV cuc are capacitor voltages, and D is grid dip severity factor. The severity of two-phase-to-ground grid fault depends on D. The range of D is [0, 1], where 1 means normal grid, and 0 means short circuit, more practices about D can be found in [20] and [21].

B. Design and Collection of Sample Data for ANN
ANN feedback is triggered when the unbalanced grid detector detects an unbalanced grid fault. After training, the ANN can give feasible circulating current references (i * cACk ) to control the SM capacitor voltages in a desired way. This letter takes twophase-to-ground grid fault as an example. The vector definition of the two-phase-to-ground grid fault can be described based on [20] asV whereV ga_pu ,V gb_pu ,V gc_pu are per unit values of the grid voltages in phase a, b, and c. Circulating current circulates inside the MMC without affecting the ac output currents. It only influences the internal performance of the MMC system. In this letter, we define the circulating current as a parameter containing both dc and ac components where k is the phase number of the three-phase system (0 for phase A, 1 for phase B, and 2 for phase C), i ck is the three-phase circulating currents, i uk and i lk are the upper and lower arm current, respectively. I ckDC is the dc component of circulating current,Î cACk cos (2ωt + θ vk− ) is the ac component of three-phase circulating current, ω is the fundamental angular frequency, and θ v− is the phase angle of negative-sequence voltage. The injected ac circulating currents are double fundamental frequency, negative-sequence phases [0, −4/3 * pi, −2/3 * pi] with different amplitudes (Î cACa ,Î cACb ,Î cACc ). In this letter, we only consider ac circulating current components.
To clarify the training procedures, all the steps are shown in Fig. 1. Each step is further elaborated as follows.  The output data was the sameV cua,b,c . The total number of data points was 1715, and the overall data collection time for 1715 data was approximately 10 min.
2) Reduction of the capacitor voltage amplitudes using ANN. The extracted data were used to train the desired surrogate model, which represents the relationship betweenV cu , D, and I cAC under two-phase-to-ground faults conditions. The trained ANN model was finally used to calculate the injected circulating current references for the MMC system under two-phase-to-ground grid faults. The inputs of the ANN model are the grid dip severity factor D and safe SM capacitor voltage limit, then the outputs of the ANN are the three-phase circulating current references. The trained ANN model was compiled to the SIMULINK model. This model can both be applied in the PLECS Blockset software (for offline simulation) and in the dSPACE DS1006 platform (for real-time control).

III. RESULTS
The control block diagram of the proposed method is shown in Fig. 2. The MMC controller is divided into two main parts. 1) Proposed ANN and circulating current controller: When two-phase-to-ground fault happens, the proposed ANN model will generate three-phase circulating current references based on desired capacitor voltages and detected D. Then, the circulating current controller enables the MMC to track its circulating current references. In this letter, the proportional resonant (PR) controller is used to track the ac circulating current [11]. The proposed ripple reduction method reduces the SM voltage ripple by injecting ac circulating current under unbalanced grid conditions. Energy controller ensures that the total amount of energy stored inside the converter is always controllable to stabilize the MMC system under unbalanced grid conditions. By energy controller, two types of energies are controlled. We refer to them as the "Sum" energy and the "Delta" energy. Sum energy is the sum of upper arm energy and lower arm energy; Delta energy is the difference between upper arm energy and lower arm energy. The detailed information about these two energy controllers can be found in [11] and [22]. The three-phase sum energies are controlled to a same level and the delta energies are controlled to zero. 2) Output current controller: PR controller is used in the output controller to control the output current of the MMC in grid-connected operation. This controller is conventional, and more details can be found in [11].

A. Simulation Results
The proposed ANN SM capacitor voltage ripple reduction method was first validated by PLECS simulation. The specifications of the simulation model are listed in Table I. The simulation results of MMC under balanced grid conditions are shown in Fig. 3. Fig. 3(a1) shows the three-phase SM capacitor voltages under balanced grid condition. These voltages are balanced and within the safe voltage limit of around 10% of the nominal value. The average voltage ripple (peak to peak value of capacitor voltage) here is 38.72 kV. The unbalanced degree (UD) is 0.046%, UD is defined as follows:   The results of the proposed method are presented in Fig. 3(c1)-(c3). The circulating current components are injected into the MMC system. Injected ac circulating currents are calculated by the ANN model, the amplitudes of them arê I cACa = 60.53 A,Î cACb = 551 A,Î cACc = 480.8 A, respectively. It can be noticed that the three-phase injected circulating currents are not symmetrical. In this way, the losses of the MMC are lower because the root mean square current in phase A is low. The capacitor voltages of the proposed method are within the safe limit, as shown in Fig. 3(c1). The average ripple is reduced to 39.87 kV, and this voltage is reduced to the same level of capacitor voltage under balanced grid conditions. The UD here is reduced to 0.64%. In this case, the operating region is extended 30% because the proposed ANN method prevents MMC capacitor overvoltage trip when D = 0.7.
In this letter, the injected ac circulating current components are flexible, which means the amplitudes of injected ac circulating current can be varied. Therefore, the three-phase components with different amplitudes usually contain positive-, negative-, and zero-sequence components. The injected circulating current components are shown in Fig. 5.

B. Experimental Results
The proposed ANN capacitor voltage reduction method is also demonstrated in an experimental setup. The experiment is carried out in a scaled-down three-phase grid-connected MMC setup with four half-bridge SMs per arm. The MMC controller is implemented in DS1006 from dSPACE. The photo of MMC experimental setup is shown in Fig. 7. The specifications of the experimental setup are also listed in Table I. The data points for experiments were collected from a detailed simulation model using the same parameters with the lab setup. In this way, the data extraction time was greatly saved.
Similar to the simulation results, three-phase capacitor voltages are balanced under balanced grid. In addition, the voltage is within the safe limit, as shown in Fig. 6(a1). The average ripple is 6.5 V, and the UD is 0.18%.
The capacitor voltages become unbalanced under two-phaseto-ground faults. Moreover, the capacitor voltage amplitudes exceed the limit, which is shown in Fig. 6(b1). The average ripple increases to 7.3 V. The UD increases to 1.112%.
The proposed method can also reduce the capacitor voltages to a safe voltage limit in the experiment, as shown in Fig. 6(c1). The amplitudes of injected ac circulating current components areÎ cACa = 0.51 A,Î cACb = 1.13 A, andÎ cACc = 1.29 A,respectively. The average ripple here is 6.5 V, and the UD is reduced to 0.5%. With the proposed method, MMC capacitor overvoltage trip is prevented when D = 0.5.

C. Extension of the Proposed Method to Other Unbalanced Grid Conditions
This letter thus far only discussed the two-phase-to-ground grid condition as an example. Here, we elaborate how this proposed method can be easily extended to other unbalanced grid conditions.
There are four types of unbalanced grid conditions [19], [20]: Two-phase-to-ground fault, three-phase-to-ground fault, single-phase-to-ground fault, and phase-to-phase short circuit fault. In this letter, the vector definition of grid under twophase-to-ground fault is expressed as (3). The vector definitions of other unbalanced grid conditions are shown in Table II. From the vector definitions, we can see that grid dip severity factor D defines the severity of the unbalanced grid, where the range of D is [0, 1]. Here, 1 means normal grid, while 0 means short circuit. By the type of unbalanced grid condition and changing D from [0, 1], we can cover all possible fault scenarios by doing numerous parallel simulations and extracting corresponding data. This process can be significantly accelerated by paralleling the tasks using a computer cluster that contains many CPU cores. Therefore, the only limitation to extend our

D. Comparison of Different MMC Operation Extension Methods
To clearly elaborate the advantages and disadvantages of the proposed machine learning method in this letter, a comparison of analytical frequency-domain based methods [4], [6], [7], the MPC oriented method [16], and the proposed machine learning method is listed in Table III.   Table III shows that the analytical frequency-domain-based methods have a low computation burden as well as a good operation region extension performance because injecting method is proven to be effective for operation region extension. However, the main drawback of these methods is that engineers need to derive the complicated analytical equations to get the desired circulating current components. The MPC oriented method is proposed in [16] to build a model predictive control to extend the operation, in this way, the parallel simulation for data collect can be eliminated. However, the nonlinear controller in this method increases the difficulty for engineers not familiar with advanced control theory to implement this method in the real MMC project.
The proposed method does not need to derive any analytical equations to get the reference of circulating current. Also, the proposed method is suitable for both balanced and unbalanced grid conditions: The proposed method uses the data from parallel simulation then trains an ML model offline to get the parameters for the desired ANN model (biases, weights). In this way, we can achieve a very low computation burden in real microprocessor such as DSP, dSPACE. Moreover, as this ML method is just an additional simple block for the traditional MMC controller (shown in Fig. 2); there is no need for engineers to build a nonlinear MMC controller from the scratch.

IV. CONCLUSION
This article proposed a fast method to reduce the capacitor voltage oscillations under two-phase-to-ground grid faults. This was achieved by training an ANN, which explicitly maps the relationships between circulating current references and capacitor voltages. The operating region was extended by 30% with the proposed method at a minimum in all testing conditions. Therefore, this article provided a solution to handle the nonparametric optimization of the complex internal characteristics of MMC. This method was validated for two-phase-to-ground grid faults but can also be extended to other unbalanced grid conditions. In future work, the application of this proposed ANN method to dc circulating current components optimization will be expanded.