The renewable energy generation has volatility and uncertainty characteristics, which can disrupt the dynamic balance of voltage and frequency in power systems, alter power flow distribution, and consequently cause grid splitting, threatening overall grid stability and power supply security. The stability control of the multiband power system stabilizer (MB-PSS) can enhance the reliability and stability of power transmission within power systems. To ensure MB-PSS stability, a Transformer-embedded deep deterministic policy gradient(TDDPG) method was proposed by combining the deep deterministic policy gradient(DDPG) method in deep reinforcement learning with the Transformer mechanism from generative pre-trained models(GPT). The Transformer mechanism was integrated into two neural networks of the DDPG method, utilizing encoders and decoders to increase the dimensionality of input parameters, addressing the issue of insufficient input in the two neural networks of the DDPG method. Simulation results of three-phase and two-phase grounding faults demonstrate that the multi-band power system stabilizer stability control method based on the TDDPG method achieves excellent training results with higher control precision.