Whatsapp down

User avatar
Beany
Posts: 6268
Joined: Wed Apr 11, 2018 5:27 pm

Re: Whatsapp down

Post by Beany »

Yeah, I saw that - that it was removed shortly afterwards (from the original Reddit source) is quite telling. Either a massive bullshitter, or realised they were leaving themselves open to a massive bollocking.

It matches the symptoms though, broadly, and the social/real world implications of the current global pandemic situation - so it's definitely an interesting comment.

Although hackernews is no more an authorative source than, say, the comments on the Register or any tech thread on Reddit, of course. The standard pinch of salt should be deployed, as ever.
DaveE
Posts: 1421
Joined: Wed Apr 11, 2018 7:19 pm

Re: Whatsapp down

Post by DaveE »

Facebook's motto is:

"Move fast and break things"

Mission accomplished.

It does seem odd though that everything is offline for everyone.

You'd think a trillion dollar company's systems would be more resilient and have better fail-over/redundancy.

I do sympathise with whomever has caused this though.

I think most of us have experience of editing autoexec.bat and getting it wrong
User avatar
Beany
Posts: 6268
Joined: Wed Apr 11, 2018 5:27 pm

Re: Whatsapp down

Post by Beany »

Well, someone put the correct Soundblaster config in there and Facebook is now back up
User avatar
Simon
Posts: 4738
Joined: Wed Apr 11, 2018 4:03 pm

Re: Whatsapp down

Post by Simon »

It's not fully backup tho. Try searching for a random name. Nothing returned.
The artist formerly known as _Who_
User avatar
Beany
Posts: 6268
Joined: Wed Apr 11, 2018 5:27 pm

Re: Whatsapp down

Post by Beany »

Why would I search for a random name on Facebook? What do you think I am, sociable? :lol:
User avatar
John
Posts: 1170
Joined: Wed Apr 11, 2018 4:31 pm

Re: Whatsapp down

Post by John »

I like to think that it was the cleaner unplugging *something* so that they could plug in their vacuum cleaner.
User avatar
Ascender
Posts: 3518
Joined: Thu Apr 12, 2018 12:07 pm
Currently Driving: 2019 M2 Competition

Re: Whatsapp down

Post by Ascender »

John wrote: Tue Oct 05, 2021 8:16 am I like to think that it was the cleaner unplugging *something* so that they could plug in their vacuum cleaner.
That's one of those legends/urban myths about the company who couldn't figure out why a server was going down at the same time every night but was also recovering itself by the time they'd got some out to look at it.
Cheers,

Mike.
User avatar
Beany
Posts: 6268
Joined: Wed Apr 11, 2018 5:27 pm

Re: Whatsapp down

Post by Beany »

Ascender wrote: Tue Oct 05, 2021 8:18 am
John wrote: Tue Oct 05, 2021 8:16 am I like to think that it was the cleaner unplugging *something* so that they could plug in their vacuum cleaner.
That's one of those legends/urban myths about the company who couldn't figure out why a server was going down at the same time every night but was also recovering itself by the time they'd got some out to look at it.
I'm sad to say I've never seen that happen in the real world :(
User avatar
240PP
Posts: 1422
Joined: Sat Apr 14, 2018 9:27 am
Currently Driving: A5 3.0 TDI, 987 S.

Re: Whatsapp down

Post by 240PP »

Ascender wrote: Tue Oct 05, 2021 8:18 am
John wrote: Tue Oct 05, 2021 8:16 am I like to think that it was the cleaner unplugging *something* so that they could plug in their vacuum cleaner.
That's one of those legends/urban myths about the company who couldn't figure out why a server was going down at the same time every night but was also recovering itself by the time they'd got some out to look at it.
I did hear the BA datacentre outage a few years ago was due to a cleaner pressing the datahall power shut off switch instead of the one that opens the door. Not sure if it’s bollocks or not.
DaveE
Posts: 1421
Joined: Wed Apr 11, 2018 7:19 pm

Re: Whatsapp down

Post by DaveE »

I used to work as a developer at Next's head office

They installed fire alarms next to door release buttons

About once a week, someone would hit the wrong button and we'd have a fleet of fire engines arrive

So 'pressing the wrong button' could certainly happen...
User avatar
Beany
Posts: 6268
Joined: Wed Apr 11, 2018 5:27 pm

Re: Whatsapp down

Post by Beany »

DaveE wrote: Tue Oct 05, 2021 11:02 am I used to work as a developer at Next's head office

They installed fire alarms next to door release buttons

About once a week, someone would hit the wrong button and we'd have a fleet of fire engines arrive

So 'pressing the wrong button' could certainly happen...
Geneeerrallly hard stop buttons have a hard cover you have to lift first.

But not always.
User avatar
Beany
Posts: 6268
Joined: Wed Apr 11, 2018 5:27 pm

Re: Whatsapp down

Post by Beany »

For those who are interested, Cloudflare have done a pretty clean writeup of what they saw, given they're one of the larger brokers of data/info/routing on the internet.

https://blog.cloudflare.com/october-202 ... ok-outage/

Also, this probably explains the internet problems (DNS) I had yesterday on EE:
But that's not all. Now human behavior and application logic kicks in and causes another exponential effect. A tsunami of additional DNS traffic follows.

This happened in part because apps won't accept an error for an answer and start retrying, sometimes aggressively, and in part because end-users also won't take an error for an answer and start reloading the pages, or killing and relaunching their apps, sometimes also aggressively.

....

So now, because Facebook and their sites are so big, we have DNS resolvers worldwide handling 30x more queries than usual and potentially causing latency and timeout issues to other platforms.
Chances are EEs public DNS servers locked into their mobile data setup got overwhelmed meaning very slow name resolution - whereas using my VPN (and consequently, not EEs DNS infrastructure but Googles and Cloudflares) meant resolution was fine as GDNS and Cloudflare DNS are massively over-provisioned for just this sort of incident.

Very interesting.
V8Granite
Posts: 3894
Joined: Fri Apr 13, 2018 11:57 am

Re: Whatsapp down

Post by V8Granite »

Can someone explain to me why not being able to message people for a few hours is a problem ?

Also the same for Facebook, I know businesses operate through it but why is half a day not being online such a big issue ?

Dave!
User avatar
Jobbo
Posts: 9175
Joined: Wed Apr 11, 2018 4:20 pm
Currently Driving: S6 Avant, Jimny, Macan, Mini

Re: Whatsapp down

Post by Jobbo »

V8Granite wrote: Tue Oct 05, 2021 2:57 pm Can someone explain to me why not being able to message people for a few hours is a problem ?

Also the same for Facebook, I know businesses operate through it but why is half a day not being online such a big issue ?

Dave!
If you weren't able to work for half a day because all your tools disappeared, would that cause you a problem? Facebook's business is solely online; they cut off their income for the period of downtime.
V8Granite
Posts: 3894
Joined: Fri Apr 13, 2018 11:57 am

Re: Whatsapp down

Post by V8Granite »

Jobbo wrote: Tue Oct 05, 2021 3:00 pm
V8Granite wrote: Tue Oct 05, 2021 2:57 pm Can someone explain to me why not being able to message people for a few hours is a problem ?

Also the same for Facebook, I know businesses operate through it but why is half a day not being online such a big issue ?

Dave!
If you weren't able to work for half a day because all your tools disappeared, would that cause you a problem? Facebook's business is solely online; they cut off their income for the period of downtime.
It would but it happens all the time. Either due to permits or the wrong permissions given, ship plans changing etc.

It really surprises me that it makes front page news.

Dave!
User avatar
Rich B
Posts: 9535
Joined: Wed Apr 11, 2018 4:22 pm
Currently Driving: M2 Competition

Re: Whatsapp down

Post by Rich B »

V8Granite wrote: Tue Oct 05, 2021 3:06 pm
Jobbo wrote: Tue Oct 05, 2021 3:00 pm
V8Granite wrote: Tue Oct 05, 2021 2:57 pm Can someone explain to me why not being able to message people for a few hours is a problem ?

Also the same for Facebook, I know businesses operate through it but why is half a day not being online such a big issue ?

Dave!
If you weren't able to work for half a day because all your tools disappeared, would that cause you a problem? Facebook's business is solely online; they cut off their income for the period of downtime.
It would but it happens all the time. Either due to permits or the wrong permissions given, ship plans changing etc.

It really surprises me that it makes front page news.

Dave!
i expect it would make news if every ship in the world suddenly changed plans at the same time.
User avatar
Jobbo
Posts: 9175
Joined: Wed Apr 11, 2018 4:20 pm
Currently Driving: S6 Avant, Jimny, Macan, Mini

Re: Whatsapp down

Post by Jobbo »

Dave, consider it as if all the telexes on the ship fail at the same time :lol:
V8Granite
Posts: 3894
Joined: Fri Apr 13, 2018 11:57 am

Re: Whatsapp down

Post by V8Granite »

It still seems like a lot of panic for such a small amount of downtime.

Dave!
User avatar
duncs500
Posts: 4568
Joined: Wed Apr 11, 2018 8:59 pm

Re: Whatsapp down

Post by duncs500 »

V8Granite wrote: Tue Oct 05, 2021 2:57 pm Can someone explain to me why not being able to message people for a few hours is a problem ?

Dave!
Well there's always SMS. ;)

I do get that's it's a problem for businesses and it is disruptive. However, there was a wedding cake maker on the radio asking what would happen to her business if it went down permanently, all I could think was that people will still need wedding cakes, so surely you'd just use whatever advertising tool replaced it?
User avatar
DeskJockey
Posts: 4629
Joined: Thu Apr 12, 2018 8:58 am

Re: Whatsapp down

Post by DeskJockey »

V8Granite wrote: Tue Oct 05, 2021 3:25 pm It still seems like a lot of panic for such a small amount of downtime.

Dave!
Think of the scale. We're talking billions of people impacted.
---
Driving a Galaxy far far away
Post Reply