Jan 14, 2015: Unpexpected outages in last 24 hours

Abhimanyu Grover
January 14, 2015

First off, we apologize to all our hosted customers for two consecutive outages in last 24 hours which resulted in downtime of 15 minutes and 60 minutes respectively.

What went wrong?
Our billing server had an unexpected issues at data center which required a manual restart – this should have been simple enough. But due to some issue, server was stuck while restart operation. Although the application server (HTTP Server + DB) was running all the time during both outages, it interacts with billing server – and when it failed to do so, many of you saw following error:
“Plan for this account has been expired. Contact your account owner”

What we’re doing to avoid this in future?
Note: As much as I’d like to put this in simple words but if you’re not a web expert you might not understand this part but we’d still like to keep things transparent with all of you.

  • Response from billing server will be cached for longer time period – so downtime on billing server won’t affect the application functioning.
  • DNS Cache times will be increased, so any failure on DNS servers won’t affect application accessibility for those with almost expiring DNS cache records.
  • We’re also considering new options to communicate with our customers during downtime.

    • ‍

    First two changes are to be implemented immediately within 48 hours.

    Once again, we apologize for this inconvenience.

    ‍

    Related Posts

    Navigating Microservices: From Refactoring and Testing
    Hong Kong 2024 Cybersecurity Legislation: What You Need to Know
    As of August 2024, Hong Kong has taken significant steps toward strengthening its cybersecurity framework, particularly focusing on critical infrastructure. This post explores the recent developments, legislative proposals, and the broader context of cybersecurity in Hong Kong based on a comprehensive research report.
    Navigating the EU AI Act in 2024: A Comprehensive Guide for Compliance
    The EU AI Act introduces comprehensive regulations for AI systems, categorizing them by risk levels and imposing corresponding compliance requirements to ensure safe and ethical AI use. Effective data governance and ethical oversight are crucial, and organizations can navigate the Act by using a detailed compliance checklist and robust test cases to validate adherence. Implementing these strategies helps businesses align with the regulation while fostering trust and transparency in their AI applications.