BSE logs out for 3 hours, hits traders - Livemint

Updated: Thu, Jul 03 2014. 09 46 PM IST

Mumbai: A network outage forced BSE Ltd to halt trading for more than three hours on Thursday, one of the longest disruptions faced by Asia’s oldest stock exchange, prompting the capital markets regulator and the finance ministry to seek an explanation from the bourse.

The problem began at 9.42am when “several users were logged out abruptly due to misbehaviour of some of the network components in the BSE network”, BSE said in a statement. Trading resumed at 12.45pm after data was shifted to backup servers at its disaster recovery site (DRC) located at the Dhirubhai Ambani Knowledge City in Navi Mumbai, the exchange said.

There was a “huge amount of trading data”, because of which the process of shifting it to the DRC took so much time, Ashishkumar Chauhan , chief executive officer of BSE, told reporters. The BSE’s benchmark Sensex closed almost flat at 25,823.75 points on Thursday.

The shutdown is a major setback for BSE as it seeks to lure investors away from larger rival National Stock Exchange Ltd (NSE) , which accounts for about 89% of the country’s derivatives market and about 93% of stock traded. Thursday’s disruption follows glitches in its trading system that have plagued the BSE since the beginning of the year.

For mission-critical processes like stock exchanges, even one minute of downtime is unaffordable, and typically an hour of downtime could impact one-eighth of trading incomes for the day, according to Naveen Mishra , research director with the data centre research team at Gartner Inc.

BSE typically sees 160-180 million orders daily, and around “one-third” of that would have got cancelled because of the shutdown, Chauhan said, declining to disclose the exact number of orders that had to be cancelled because of the shutdown.

The BSE network, according to the statement, connects to more than 10,000 primary connections with all telecom vendors in India participating in the network, including the exchange’s own LAN (local area network) and VSAT (very small aperture terminal) network provided by Hughes Communications.

More than 8,000 connections log in daily. Each primary connection can potentially have scores of users to several thousands, depending on member convenience and their own network configurations, the statement added.

“An attempt was being made to shift to the disaster recovery site, but you need to give sufficient time for the remedial measures as the complete trading data needs to be shifted to the new site. Even if you look at global exchanges, the shift to DRC is not instant because these are complex systems with huge amount of real-time data,” said Chauhan.

According to a 22 June 2012 circular on guidelines for business continuity plan and disaster recovery by the Securities and Exchange Board of India (Sebi), downtime cannot exceed four hours and 30 minutes. The market regulator has sought a detailed report from BSE on the trading glitch.

“Sebi and the ministry of finance have asked for the details and the root cause analysis. The process is to prepare the report in a particular format, then present it to the technology advisory committee. Then the BSE board takes it up and submits the report to the regulatory body,” said Chauhan, adding that vendors are helping it do a complete analysis to examine whether the shutdown was a result of the failure of a hardware equipment or technology or a combination of both.

“After checking with our other customers today who also use our VSATs, nobody else experienced any issues, and so the problem at BSE today was not at our end,” said Shivaji Chatterjee , vice-president of enterprise of Hughes Communications , the company that provides 900 VSATs to BSE.

Since Sebi also mandates that both the BSE and NSE have a very high standard of compliance and a very strong disaster recovery plan, “BSE also has a shadow site in Hyderabad, but at times there are issues in switching from the primary to secondary site, especially if the issue is not a major disaster, which is when it takes longer to fix the issue”, Chatterji added.

Most leading enterprises go in for business continuity solutions and keep running pilots, which to a large extent considerably reduces the downtime window, said Mishra of Gartner.

“From time to time, enterprises go in for planned downtime, like on holidays, or when the site is closed. However, for unplanned downtime, the time to get up-and-running depends on the nature of the problem and its severity. Typically, it would take anywhere between 10 minutes to 12 hours to get a problem rectified,” said Mishra.

Every business needs a formal, documented business continuity or risk-intelligent plan that can anticipate risks, and periodically revisit them to ensure business continuity, said Sanjoy Sen, senior director of Deloitte Touche Tohmatsu India Pvt. Ltd, a global consulting firm.

“This helps in proactively building resilience, instead of just reacting to it. Large global organizations, especially in the financial services, healthcare and life sciences and telecom space, which have mission-critical applications, all have business continuity plans in place—even for their Indian arms,” he said.