OV9

Posted: **Mon Oct 04, 2021 10:00 pm**

Yeah, I saw that - that it was removed shortly afterwards (from the original Reddit source) is quite telling. Either a massive bullshitter, or realised they were leaving themselves open to a massive bollocking.

It matches the symptoms though, broadly, and the social/real world implications of the current global pandemic situation - so it's definitely an interesting comment.

Although hackernews is no more an authorative source than, say, the comments on the Register or any tech thread on Reddit, of course. The standard pinch of salt should be deployed, as ever.

Posted: **Mon Oct 04, 2021 10:44 pm**

Facebook's motto is:

"Move fast and break things"

Mission accomplished.

It does seem odd though that everything is offline for everyone.

You'd think a trillion dollar company's systems would be more resilient and have better fail-over/redundancy.

I do sympathise with whomever has caused this though.

I think most of us have experience of editing autoexec.bat and getting it wrong

Posted: **Mon Oct 04, 2021 11:18 pm**

Well, someone put the correct Soundblaster config in there and Facebook is now back up

Posted: **Mon Oct 04, 2021 11:33 pm**

It's not fully backup tho. Try searching for a random name. Nothing returned.

Posted: **Mon Oct 04, 2021 11:53 pm**

Why would I search for a random name on Facebook? What do you think I am, sociable?

Posted: **Tue Oct 05, 2021 8:16 am**

I like to think that it was the cleaner unplugging *something* so that they could plug in their vacuum cleaner.

Posted: **Tue Oct 05, 2021 8:18 am**

John wrote: Tue Oct 05, 2021 8:16 am I like to think that it was the cleaner unplugging *something* so that they could plug in their vacuum cleaner.

That's one of those legends/urban myths about the company who couldn't figure out why a server was going down at the same time every night but was also recovering itself by the time they'd got some out to look at it.

Posted: **Tue Oct 05, 2021 9:34 am**

Ascender wrote: Tue Oct 05, 2021 8:18 am
John wrote: Tue Oct 05, 2021 8:16 am I like to think that it was the cleaner unplugging *something* so that they could plug in their vacuum cleaner.
That's one of those legends/urban myths about the company who couldn't figure out why a server was going down at the same time every night but was also recovering itself by the time they'd got some out to look at it.

I'm sad to say I've never seen that happen in the real world

Posted: **Tue Oct 05, 2021 10:30 am**

Ascender wrote: Tue Oct 05, 2021 8:18 am
John wrote: Tue Oct 05, 2021 8:16 am I like to think that it was the cleaner unplugging *something* so that they could plug in their vacuum cleaner.
That's one of those legends/urban myths about the company who couldn't figure out why a server was going down at the same time every night but was also recovering itself by the time they'd got some out to look at it.

I did hear the BA datacentre outage a few years ago was due to a cleaner pressing the datahall power shut off switch instead of the one that opens the door. Not sure if it’s bollocks or not.

Posted: **Tue Oct 05, 2021 11:02 am**

I used to work as a developer at Next's head office

They installed fire alarms next to door release buttons

About once a week, someone would hit the wrong button and we'd have a fleet of fire engines arrive

So 'pressing the wrong button' could certainly happen...

Posted: **Tue Oct 05, 2021 11:33 am**

DaveE wrote: Tue Oct 05, 2021 11:02 am I used to work as a developer at Next's head office

They installed fire alarms next to door release buttons

About once a week, someone would hit the wrong button and we'd have a fleet of fire engines arrive

So 'pressing the wrong button' could certainly happen...

Geneeerrallly hard stop buttons have a hard cover you have to lift first.

But not always.

Posted: **Tue Oct 05, 2021 2:00 pm**

For those who are interested, Cloudflare have done a pretty clean writeup of what they saw, given they're one of the larger brokers of data/info/routing on the internet.

https://blog.cloudflare.com/october-202 ... ok-outage/

Also, this probably explains the internet problems (DNS) I had yesterday on EE:

But that's not all. Now human behavior and application logic kicks in and causes another exponential effect. A tsunami of additional DNS traffic follows.

This happened in part because apps won't accept an error for an answer and start retrying, sometimes aggressively, and in part because end-users also won't take an error for an answer and start reloading the pages, or killing and relaunching their apps, sometimes also aggressively.

....

So now, because Facebook and their sites are so big, we have DNS resolvers worldwide handling 30x more queries than usual and potentially causing latency and timeout issues to other platforms.

Chances are EEs public DNS servers locked into their mobile data setup got overwhelmed meaning very slow name resolution - whereas using my VPN (and consequently, not EEs DNS infrastructure but Googles and Cloudflares) meant resolution was fine as GDNS and Cloudflare DNS are massively over-provisioned for just this sort of incident.

Very interesting.

Posted: **Tue Oct 05, 2021 2:57 pm**

Can someone explain to me why not being able to message people for a few hours is a problem ?

Also the same for Facebook, I know businesses operate through it but why is half a day not being online such a big issue ?

Dave!

Posted: **Tue Oct 05, 2021 3:00 pm**

V8Granite wrote: Tue Oct 05, 2021 2:57 pm Can someone explain to me why not being able to message people for a few hours is a problem ?

Also the same for Facebook, I know businesses operate through it but why is half a day not being online such a big issue ?

Dave!

If you weren't able to work for half a day because all your tools disappeared, would that cause you a problem? Facebook's business is solely online; they cut off their income for the period of downtime.

Posted: **Tue Oct 05, 2021 3:06 pm**

Jobbo wrote: Tue Oct 05, 2021 3:00 pm
V8Granite wrote: Tue Oct 05, 2021 2:57 pm Can someone explain to me why not being able to message people for a few hours is a problem ?

Also the same for Facebook, I know businesses operate through it but why is half a day not being online such a big issue ?

Dave!
If you weren't able to work for half a day because all your tools disappeared, would that cause you a problem? Facebook's business is solely online; they cut off their income for the period of downtime.

It would but it happens all the time. Either due to permits or the wrong permissions given, ship plans changing etc.

It really surprises me that it makes front page news.

Dave!

Posted: **Tue Oct 05, 2021 3:09 pm**

V8Granite wrote: Tue Oct 05, 2021 3:06 pm
Jobbo wrote: Tue Oct 05, 2021 3:00 pm
V8Granite wrote: Tue Oct 05, 2021 2:57 pm Can someone explain to me why not being able to message people for a few hours is a problem ?

Also the same for Facebook, I know businesses operate through it but why is half a day not being online such a big issue ?

Dave!
If you weren't able to work for half a day because all your tools disappeared, would that cause you a problem? Facebook's business is solely online; they cut off their income for the period of downtime.
It would but it happens all the time. Either due to permits or the wrong permissions given, ship plans changing etc.

It really surprises me that it makes front page news.

Dave!

i expect it would make news if every ship in the world suddenly changed plans at the same time.

Posted: **Tue Oct 05, 2021 3:15 pm**

Dave, consider it as if all the telexes on the ship fail at the same time

Posted: **Tue Oct 05, 2021 3:25 pm**

It still seems like a lot of panic for such a small amount of downtime.

Dave!

Posted: **Tue Oct 05, 2021 3:43 pm**

V8Granite wrote: Tue Oct 05, 2021 2:57 pm Can someone explain to me why not being able to message people for a few hours is a problem ?

Dave!

Well there's always SMS.

I do get that's it's a problem for businesses and it is disruptive. However, there was a wedding cake maker on the radio asking what would happen to her business if it went down permanently, all I could think was that people will still need wedding cakes, so surely you'd just use whatever advertising tool replaced it?

Posted: **Tue Oct 05, 2021 6:56 pm**

V8Granite wrote: Tue Oct 05, 2021 3:25 pm It still seems like a lot of panic for such a small amount of downtime.

Dave!

Think of the scale. We're talking billions of people impacted.

OV9

Whatsapp down

Re: Whatsapp down

Re: Whatsapp down

Re: Whatsapp down

Re: Whatsapp down

Re: Whatsapp down

Re: Whatsapp down

Re: Whatsapp down

Re: Whatsapp down

Re: Whatsapp down

Re: Whatsapp down

Re: Whatsapp down

Re: Whatsapp down

Re: Whatsapp down

Re: Whatsapp down

Re: Whatsapp down

Re: Whatsapp down

Re: Whatsapp down

Re: Whatsapp down

Re: Whatsapp down

Re: Whatsapp down