Page 1 of 1

AWS

Posted: Mon Oct 20, 2025 10:06 am
by IanF
Are we blaming @Beany or @Simon ..

/insert Airport gif here

Re: AWS

Posted: Mon Oct 20, 2025 10:09 am
by Beany
It's US EAST again by the looks

https://news.ycombinator.com/item?id=45640772

Which you'd think would be fine if you're using Zoom, who IIRC host their EU instances in AWS EU EAST or whatever.

However, if htey use Cloudfront, S3, or one of a dozen other AWS internal services, those are primarily hosted and controlled from AWS US EAST so, er, good luck I guess.

Our Zoom is out, and our Slack is a bit 'sticky' and slow, but luckily I'm off today (which I only realised ten minutes into our morning standup) so I class this as not my fucking problem bwaaahaahahahahahaha etc, which my colleagues appreciated 8-)

Re: AWS

Posted: Mon Oct 20, 2025 10:17 am
by Mito Man
Ring cameras have gone loopy.

Re: AWS

Posted: Mon Oct 20, 2025 10:18 am
by Beany
Mito Man wrote: Mon Oct 20, 2025 10:17 am Ring cameras have gone loopy.
Probably S3 or DNS/CDN hosted in AWS US EAST

Scroll through this, everything is fucked.

https://downdetector.co.uk/

Re: AWS

Posted: Mon Oct 20, 2025 10:24 am
by Mito Man
Just got alerts again and everything is back!

Re: AWS

Posted: Mon Oct 20, 2025 10:30 am
by Jobbo
I rather like all wheel steering now.

Re: AWS

Posted: Mon Oct 20, 2025 10:37 am
by Beany
Slack on my work phone hasn't caught up to reality (IE the messages I sent from the laptop earlier haven't appeared in it) so I'm guessing the problem is still ongoing for the most part, but some services might have failed over to other providers, etc.

Edit: apparently AWS DNS is back up so it smells like they're getting services back up and running piece by piece.

Re: AWS

Posted: Mon Oct 20, 2025 11:08 am
by Gavster
Ring doorbells still down here, simply hasn't been able to connect for the last three hours

Re: AWS

Posted: Mon Oct 20, 2025 11:11 am
by Beany
This morning feels like this for me.


Re: AWS

Posted: Mon Oct 20, 2025 11:47 am
by Mito Man
And it’s down again.

Re: AWS

Posted: Mon Oct 20, 2025 12:02 pm
by Beany
Yeah I'd expect things to be a bit up and down, as AWS themselves try to drag their services kicking and screaming back into life, and as other orgs (your Zooms, your PSNs etc) go between their fallback setups if they have them, and back to AWS on prod, etc.

Basically, as a professional Head of IT, I demand you all use this as an excuse to bunk off a bit today - you've all earned it. 8-)

"Nah mate, my Zoom still ain't working, lets reschedule for next week" you cry, as you sit in your pants and scratch your nuts on the couch....

...in the office breakroom.

Re: AWS

Posted: Mon Oct 20, 2025 1:03 pm
by Rich B
Yep, my company uses autodesk construction cloud and Autodesk Revit for all our drawings - good to know how COMPLETELY fucked our company is without one provider.

Re: AWS

Posted: Mon Oct 20, 2025 1:13 pm
by Simon
Fun fact, did you know the Amazon website actually uses a competitor <cough> to AWS Cloudfront? That's why the Amazon retail site wasn't so affected this morning. :-D

Re: AWS

Posted: Mon Oct 20, 2025 2:41 pm
by Jobbo
Image