Service Disruption Post Mortem - August 31st - The Bitfinex Blog
17223
post-template-default,single,single-post,postid-17223,single-format-standard,bridge-core-3.0.6,et_bloom,qode-page-transition-enabled,ajax_fade,page_not_loaded,,qode-title-hidden,qode_grid_1300,footer_responsive_adv,qode-content-sidebar-responsive,qode-child-theme-ver-1.0.0,qode-theme-ver-29.3,qode-theme-bridge,qode_header_in_grid,wpb-js-composer js-comp-ver-6.10.0,vc_responsive

Service Disruption Post Mortem – August 31st

On Friday 31st August, 2018, the Bitfinex platform encountered an issue which forced us to suspend trading for a total of roughly 60 minutes. At 14.55 UTC the platform started encountering issues originating from our cloud hosting provider restarting a core matching server.

Our team begun assessing the damage immediately. Here we ensured that no data was lost and we flushed the pending queues to the database.

Once the integrity of the system was established, the restart and resync procedure was performed, as well as a series of automated and manual tests to ensure all functionalities were in order.

A list of all executed trades during downtime can be found here.

Please note in the above file that Type: Synchronising ME refers to trades executed as we recovered data that was not fully synced and saved onto our databases.

All users were kept informed via twitter.com/bitfinex, as well as bitfinex.statuspage.io. We are taking concrete steps to avoid a recurrence of this type of event and we sincerely apologise for the impact of our unplanned downtime.

The Bitfinex team

 


 

Stay up to date with Bitfinex on TwitterLinkedInTelegram, Facebook & YouTube

The Bitfinex APIs are designed to allow complete access to the features provided by Bitfinex. Learn more about our API documentation here.

We’re hiring! Check out our open positions and get in touch.