Peter Zhang
Nov 21, 2024 22:28
Sui Community skilled a short outage on account of a bug in congestion management. Swift motion by engineers and validators restored operations inside minutes, highlighting efficient incident response.
The Sui Mainnet lately confronted a big outage, halting all community operations for a number of hours on account of a technical glitch. The incident, which occurred on November 21, 2024, between 1:15 and three:45 am PT, concerned a crash loop affecting all validators, stopping any transaction processing, in line with the Sui Basis.
Understanding the Incident
The problem stemmed from a bug within the congestion management code, particularly an assert! assertion, which triggered a crash when the estimated execution price was zero. This downside was linked to the TotalGasBudgetWithCap mode, briefly enabled in protocol model 63 and reintroduced in model 68. The bug manifested when the community acquired a transaction with a mutable shared object enter and 0 MoveCall instructions, inflicting all validators to crash.
The Function of Congestion Management
Congestion management within the Sui community is essential for managing transaction charges to shared objects, making certain the community doesn’t change into overloaded. This technique was lately upgraded to boost shared object utilization by precisely estimating transaction complexity. Nonetheless, the improve inadvertently launched the bug inflicting the outage.
Decision and Response
Upon figuring out the issue, Sui engineers promptly devised a repair. The corrective code, detailed in PR #20365, was deployed to each the Mainnet and Testnet in variations v1.37.4 and v1.38.1, respectively. The fast deployment was facilitated by an impressive response from the validator group, enabling the community to renew operations inside quarter-hour of releasing the repair.
Classes and Future Enhancements
This incident underscored the effectiveness of Sui’s incident detection and response mechanisms. Automated alerts promptly notified engineers, who collaborated with the validator group to handle the difficulty swiftly. Transferring ahead, Sui plans to boost its testing methods to forestall comparable bugs and streamline its construct workflows to scale back incident response instances.
For extra detailed info, please go to the Sui Basis.
Picture supply: Shutterstock