Kiln Website is unavailable for 30 customers KILN-1507
Incident Report for FogBugz and Kiln On Demand
Resolved
Outage has been resolved, outage duration was 3:51.
All affected repositories are back online.
Posted about 1 month ago. Dec 14, 2018 - 12:14 EST
Update
Problems identified:
30 customers can't access their repositories
Actions taken:
Team has fixed the issue for 15 customers. Fixing the remaining 15 ones.
Estimated duration: 60 minutes.
Posted about 1 month ago. Dec 14, 2018 - 11:53 EST
Update
Problems identified:
30 customers can't access their repositories
Actions taken:
36 customers were impacted. Team has already fixed 2 and working on fixing the remaining ones.
Estimated duration: 30 minutes.
Posted about 1 month ago. Dec 14, 2018 - 11:09 EST
Update
Problems identified:
30 customers can't access their repositories
Actions taken:
Root cause was identified and the issue for one customer was fixed. Working on fixing remaining ones
Estimated duration: 30 minutes.
Posted about 1 month ago. Dec 14, 2018 - 10:54 EST
Update
Problems identified:
30 customers can't access their repositories
Actions taken:
Team is checking Redis database.
Estimated duration: 30 minutes.
Posted about 1 month ago. Dec 14, 2018 - 10:37 EST
Update
Problems identified:
30 customers can't access their repositories
Actions taken:
Team is checking the communication between the web servers. Investigation is undergoing.
Estimated duration: 30 minutes.
Posted about 1 month ago. Dec 14, 2018 - 10:00 EST
Update
Impact confirmed to be multiple customers. We are working on identifying the scope.
Posted about 1 month ago. Dec 14, 2018 - 09:35 EST
Update
Problems identified:
30 customers can't access their repositories
Actions taken:
This seems to be a particular issue as no other reports were received. Checking system and services.
Estimated duration: 30 minutes.
Posted about 1 month ago. Dec 14, 2018 - 09:34 EST
Update
We are continuing to work on a fix for this issue.
Posted about 1 month ago. Dec 14, 2018 - 09:14 EST
Update
Problems identified:
Kiln has performance issues
Actions taken:
Team is in a call checking what might have caused this issue.
Estimated duration: 60 minutes.
Posted about 1 month ago. Dec 14, 2018 - 09:09 EST
Identified
Starting December 14, 2018 13:20 GMT we are experiencing an incident: Kiln Website is unavailable. This is affecting restricted set of customers.
We are working on the resolution and expect to have it resolved within the next 60 minutes.

We will provide an update every 30m or as soon as incident is resolved.
Posted about 1 month ago. Dec 14, 2018 - 08:28 EST
This incident affected: Kiln Website.