Software error to blame for two-hour Hong Kong MTR stoppage that left tens of thousands stranded

An investigation into the worst service interruption on the MTR’s East Rail Line in years has concluded that a software coding error was to blame for a control system failure that left tens of thousands of commuters stranded.

On the morning of January 11, the entire vital rail link between the eastern New Territories and Kowloon was suspended from about 9.30am for more than two hours.

The inquiry confirmed that the fault arose due to a hidden software coding error in the train control system (TCS) software module related to a specific command, an MTR spokesman said on Monday, after the operator submitted the investigation results to the government.

The suspension saw 14 trains stuck in between stations.

Passengers on two trains in Fanling and Fo Tan were seen forcing the doors open and walking along the tracks to get to a station.

Shuttle buses were provided between Kowloon Tong and Lo Wu, but there was a backlog of passengers at many stations.

“The adverse effect of the coding error emerged to impact service only after traffic demand continuously increased over the years,” the spokesman said.

The train regulation command was applied right before the incident, triggering a series of errors that resulted in the halting of all workstations and the TCS server.

The spokesman said the system had automatically and immediately switched over to a default standby server, but it failed to recover. The system eventually resumed normal operations after the TCS fallback server and its workstations were activated manually.

The incident caused a suspension of the East Rail Line train service for 122 minutes, the operator confirmed.

The corporation said improvement measures had been taken, including the downloading of a new software patch that rectified the coding error, as well as the enhancement of the recovery procedures.

The investigation concluded that the handling of passengers in trains, stations and free shuttle bus stops was “safe and orderly” that morning, when an additional 320 staff were deployed to affected stations. There were 131 free shuttle buses running along the affected stations, and more than 16,400 passengers were served.

It also found that the incident was not related to signalling replacement works being carried out on the line.

The East Rail Line recorded an hourly average of 56,800 passenger trips during the morning peak in 2016.