Bye bye Theresa

[Simon]

Just to be clear, here's what Unboxd (those who do the technical work for the petitions site) had to say about the last big push on the petitions site.

It covers literally everything that Julia Hartley-Brewer - who is undoubtedly the laughably misinformed idiot pushing these conspiracy theories and where G.Gs info originated from, in some form - pushed and rubbishes it.

This was over two years ago, by the way, and they shored the service up then. The current eyes on the petition has managed to break even that from a performance standpoint.

https://technology.blog.gov.uk/2016/08/ ... eferendum/

1.2million 'not from the uk'

The other change we made was related to a feature that tracks signatures by country. The vast majority of signatures coming from the UK caused a database bottleneck so we had to disable UK signatures from being recorded (in the signature counts per country statistics) while we fixed the problem. The country was still recorded on each individual signature and the signature was still recorded in the overall count during this period. The API published an incorrect count for UK signatures for a while but this count isn’t exposed by the main public website, and our petition map website, which displays signatures by constituency, wasn't affected.

'multiple email accounts'/email fraud/throwaway domains:

Since 90% of our emails are sent to large providers like Google, Apple and Microsoft that have their own anti-abuse measures, we are generally confident that these are valid accounts. This assurance allowed us to focus our search on emails coming from so called ‘disposable domains’, which are temporary email accounts that only exist for short periods before being discarded and can be created by scripts. We have a list of these domains but new ones are created constantly so it was just a matter of checking for these new domains.

Another warning sign is large numbers of signatures coming from the same IP address and this coupled with the domain checking allowed us to invalidate around 30,000 signatures that purportedly came from the Vatican.

We had similar reports about signatures coming from Bracknell. By the time the reports were published, we’d already removed the signatures as we were also checking the predicted number of signatures for each constituency compared to those we observed (an analyst also confirmed independently the signatures were fake).

The rest of that page is worth a read. The petitions site was never some rinky-dink setup - it's pretty solid, and currently under load that had never even been considered, even after the 2016 petition which was also unprecedented - which should give an idea of just how many people are clicking the link.

Sorry Beany, there's a hell of a lot in that blog I disagree with and they've basically doing things wrong. And yes, I work in this industry, yes I deal with sites at massive scale and yes that load could have been both a) tested for and b) sustained without issue.

[NotoriousREV]

[Marv]

China - 71
Russia - 28

Seems a bit low, I'd expect them to be in the millions

[NotoriousREV]

Sorry Beany, there's a hell of a lot in that blog I disagree with and they've basically doing things wrong. And yes, I work in this industry, yes I deal with sites at massive scale and yes that load could have been both a) tested for and b) sustained without issue.

The DB design must be terrible. They don’t capture that much information from you when you sign, so each DB write shouldn’t be all that big.

Although I think Simon is either being a little bit harsh or he doesn’t understand what the word “unprecedented” means

[Simon]

Dave, trust me on this. We 'do' unprecedented. That's our job!

OK, so full disclosure here. I work for Akamai, which doesn't just operate in the Content Delivery industry, but we literally invented it a little over 20 years ago. I've been here 5 years, first in our UKI team of Professional Services and now in technical pre-sales. I know a lot about our platform, the industry as a whole, and how you scale to meet demand. We deliver the biggest traffic volumes of anyone in the industry (in Europe we're bigger than all our competitors combined), and on a typical day deliver about 25-30% of all web content. I won't discuss specific customers apart from where that information is already in the public domain.

We also have the worlds largest load-testing platform, have a rapidly growing security division protecting both data-centres (think of the GitHub 1.3Tb per second attack which we shrugged off) along with a WAF that's present in all of our near-275,000 servers across the world. We operate both carrier recursive DNS servers that answer billions of DNS queries per day, and an authoritative DNS service for our own customers. Finally, as of the last couple of months we're now a leader in customer identity and access management thanks to our acquisition of JanRain, looking after some customers with over a billion customer identities.

My main disagreement is the authors presumption that using a 'CDN' won't help him scale because the site is largely transactional. Whilst that may be true of pure-play caching CDN like Cloudflare or Fastly, we have technology that can move a lot of their business logic to the Edge. So for example we can deal with the Bots first and foremost. We're able to identify and manage (note, not necessarily block) bots with a high degree of accuracy using advanced machine learning, device heuristics, device sensor data (in the case of mobile phones) and our own IP reputational database (because on any day we see just about every IP on the internet, so can build up profiles of which are good or bad). Then the main petition page itself could be cached on the Edge (even the signature count) with a low TTL because it's not being dynamically updated. We could load balance across multiple origins and manage that session stickiness for them. Yada yada yada.

As of this week I’m internally certified to demonstrate load testing, and I was examined by our internal guys who did the load testing for the Olympics. Think 500,000 simultaneous user sessions. I could go on but I won’t, although I agree with your assertion that the DB design must be poor.

[Foz]

Looking for a hard pill to swallow?

Wow

[Richard]

Dave, trust me on this. We 'do' unprecedented. That's our job!

OK, so full disclosure here. I work for Akamai, which doesn't just operate in the Content Delivery industry, but we literally invented it a little over 20 years ago. I've been here 5 years, first in our UKI team of Professional Services and now in technical pre-sales. I know a lot about our platform, the industry as a whole, and how you scale to meet demand. We deliver the biggest traffic volumes of anyone in the industry (in Europe we're bigger than all our competitors combined), and on a typical day deliver about 25-30% of all web content. I won't discuss specific customers apart from where that information is already in the public domain.

We also have the worlds largest load-testing platform, have a rapidly growing security division protecting both data-centres (think of the GitHub 1.3Tb per second attack which we shrugged off) along with a WAF that's present in all of our near-275,000 servers across the world. We operate both carrier recursive DNS servers that answer billions of DNS queries per day, and an authoritative DNS service for our own customers. Finally, as of the last couple of months we're now a leader in customer identity and access management thanks to our acquisition of JanRain, looking after some customers with over a billion customer identities.

My main disagreement is the authors presumption that using a 'CDN' won't help him scale because the site is largely transactional. Whilst that may be true of pure-play caching CDN like Cloudflare or Fastly, we have technology that can move a lot of their business logic to the Edge. So for example we can deal with the Bots first and foremost. We're able to identify and manage (note, not necessarily block) bots with a high degree of accuracy using advanced machine learning, device heuristics, device sensor data (in the case of mobile phones) and our own IP reputational database (because on any day we see just about every IP on the internet, so can build up profiles of which are good or bad). Then the main petition page itself could be cached on the Edge (even the signature count) with a low TTL because it's not being dynamically updated. We could load balance across multiple origins and manage that session stickiness for them. Yada yada yada.

As of this week I’m internally certified to demonstrate load testing, and I was examined by our internal guys who did the load testing for the Olympics. Think 500,000 simultaneous user sessions. I could go on but I won’t, although I agree with your assertion that the DB design must be poor.

Yawn

[Richard]

“I’ve designed Ferrari’s which can go over 200mph, so I think you’ll agree I come from a platform of expertise when I say that I think it’s laughable that a Honda Jazz designed to get to 100mph can’t go 200mph!”

[RobYob]

Looking for a hard pill to swallow?

Wow

Brilliant. If nothing else, Brexit has reinvigorated Britsh political comedy.

[NotoriousREV]

<stuff>

Moving logic to the edge won’t remove a DB bottleneck; they don’t have an issue at the app layer. Using a CDN to mask poor performance is a fools errand.

Load testing doesn’t help you if you don’t test for the unprecedented load. I also find load testing gives a false sense of security and never mimics real-life traffic in a way that helps you; the stress always turns up where you weren’t looking anyway. Nothing beats having easily scaleable infrastructure and apps.

When you and I are no longer working at our current employers, you and I are going to have a very interesting chat

[Rich B]

How can you test for something unprecedented?



(I don’t need an angry essay reply, I’m only joking - I put a smilie and everything!)

[GG.]

Ok so i missed all these posts last night

Fair play if the website is more robust (it is a government site so I had low expectations!) and re the multiple tries with one address point - I did effectively clarify that in my original post but the point remains that you can still use multiple valid email addresses for one person and just change the petitioner name on the form

Anyway, that issue aside, I don’t think anyone responded to the point about clarksons 1m signature. If Jezza got to that level for punching someone in the gob over a row about cold food, then actually they numbers you’re looking at on this petition are totally unremarkable given the size of the issue and how long it has been the focus of public attention.

As many people have said, if it gets into the tens of millions with adjustments made to remove people who are obviously not part of the franchise (ie all overseas including British expats, as per the original referendum) then it would move the dial.

[Mito Man]

I signed yesterday morning and still haven’t got a confirmation email.

[Rich B]

I signed yesterday morning and still haven’t got a confirmation email.

mine went into junk.

[Mito Man]

I signed yesterday morning and still haven’t got a confirmation email.

mine went into junk.

Ah must have missed it in there the first time. Thank you for reminding me and making me a millionaire at the same time because I just got this

“Prior to the payment release order submitted to JP Morgan Chase Bank, NY. I hereby write to bring to your notice regarding the release of your fund that you are entitled to receive your approved fund valued US$5,500,000.00 through the method of swift bank wire transfer system.”

Off to plan to my supercar purchases suckers

[240PP]

$5.5m isn’t going to get you into a McLaren F1.

#disappointing

[Simon]

<stuff>

Moving logic to the edge won’t remove a DB bottleneck; they don’t have an issue at the app layer. Using a CDN to mask poor performance is a fools errand.

Load testing doesn’t help you if you don’t test for the unprecedented load. I also find load testing gives a false sense of security and never mimics real-life traffic in a way that helps you; the stress always turns up where you weren’t looking anyway. Nothing beats having easily scaleable infrastructure and apps.

When you and I are no longer working at our current employers, you and I are going to have a very interesting chat

The very point of load testing is to identify when your application stack breaks, identify individual weak points in the whole architecture and also to correlate user numbers against performance counters. We get around the 'mimic'ing of live traffic' issue by actually using actual real user journeys from our RUM tool. No sampling, this is 100% of user journeys that are captured over a relevant period of time and then the load test is built from those journeys, so it's as realistic as possible. And you're able to then scale the load to whatever number you can think of by replaying those sessions in multiple 'tracks' and from multiple locations. We can also either hook into any of the major APMs on the market or supply our own agent to monitor the origin stack so you can see how the elements that support the site are responding individually.

Yes, having an elastic infrastructure is important, but more than often nowadays we find that those cloud providers don't scale quickly enough to deal with, for example, a flash crowd. I've seen Azure take 45 minutes or so to scale for one of my customers, which is useless when you've got a peak traffic from a TV advert or TV program sponsorship.

If you want to discuss anything in detail feel free to PM me.

[NotoriousREV]

I’ve been in this game for a long time (20 years) and I’ve never seen a site fail where load testing says it will.

[ZedLeg]

The forum confirms Rev’s skillz

A06CCC24-7CD3-4221-8D5F-1542B3DDCEA9.jpeg (97.75 KiB) Viewed 2978 times

[Simon]

I’ve been in this game for a long time (20 years) and I’ve never seen a site fail where load testing says it will.

Then the load testing wasn't done right

Anyway, back to politics. 2nd Ref yay or nay?

And I see that Maybot was suggesting that she wouldn't have MV3 if it didn't have support in the house. FFS woman, they've said we'll only get the extensions if we have the vote! Get it out the way, see it rejected and let's have the indicative votes on the alternatives. Then choose the one that has most support and submit that as our new play to the EuCo. It's really not hard.