Incidents reported on status page for Capptions BV https://status.capptions.com/ https://d1lppblt9t2x15.cloudfront.net/logos/d754f0ccede94d58e96306d763642136.png https://status.capptions.com/ en https://status.capptions.com/ Mon, 23 Jun 2025 11:06:41 +0000 https://status.capptions.com/#e33af03f8c85a44663d9590f0a05bbc205df3892b1c199069ddf0302d3ba6520 Capptions Web Applications recovered https://status.capptions.com/ Mon, 23 Jun 2025 11:04:41 +0000 https://status.capptions.com/#e33af03f8c85a44663d9590f0a05bbc205df3892b1c199069ddf0302d3ba6520 Capptions Web Applications went down https://status.capptions.com/incident/248547 Sat, 19 Aug 2023 12:29:00 -0000 https://status.capptions.com/incident/248547#b549f57decf19fec7552e8a7f3600c401affe4970ee58a773f6beb01fbc53329 Every year in July or August, the main SSL certificate for capptions.com and other assets needs to be renewed. Since 2021 certificates are managed through our central infrastructure repository and automatically deployed through Terraform. To make sure that the renewal process proceeds without any glitches or outages we do have the following reminders in place: - Automated reminders from our certificate provider - Calendar based reminder for manual check Both reminders were acknowledged and we did internally verify the working of the renewed certificates. However on the morning of August 19th 2023 we started getting notifications from a few customers that they couldn't log in or that there were issues related to uploading of images. After reviewing the situation we found that for the following assets the certificate renewal were excluded from the automated process and the checks were insufficient: 1. cdn.capptions.com, serving static assets including images and scripts for the login pages, causing the main login pages not to work and the web apps not to render 2. download.capptions.com, serving customer uploaded media, causing assets not to load or potentially breaking PDF generation The main point of regret here is users not being able to use our web apps on Saturday morning (CET) and our health monitors not catching this issue. Luckily after the first client reports we could swiftly recover the systems and have everything fully operational again within one hour. We are now taking steps to prevent this issue from happening again by: - Making sure the manual checks will catch these issues - Fixing the automatic certificate deployment for mentioned assets - Extending our automatic monitors to catch similar issues