The AI Thread

[DeskJockey]

It happens a lot, hence why any remotely responsible org uses it as a first pass, but no more. Verify everything, and get better at writing the prompts.

[Beany]

It happens a lot, hence why any remotely responsible org uses it as a first pass, but no more. Verify everything, and get better at writing the prompts.

And have that in a policy, attached to your disciplinary process - that way if someone does (on a smaller scale) ignore it, you can actually do something about it.

[Simon]

It happens a lot, hence why any remotely responsible org uses it as a first pass, but no more. Verify everything, and get better at writing the prompts.

Absolutely. Prompt integrity is everything. I had Gemini knock my up a shopping list for a DIY project I'm doing at the moment. When I came to do the order I had a question about one of the products and it replied with basically "oh my bad, that's completely the wrong thing". Sorry. You want the complete opposite of that product. Turns out that it was a misunderstanding based on the questions I'd been asking, but not explicitly enough.

[Beany]

In fun AI news, the game Subnautica 2 was delayed in weird and deeply sus ways by the publisher, Krafton, including sacking their founders etc.

Turns out, yes as suspected it was a laughable breach of contract. Turns out the publishers CEO got legal advice from ChatGPT, who told him "No, you can't get out of paying them up to $250m in bonuses if the game performs as expected, that'd be breach of contract"

And the CEO asked GPT "Yeah, but what if I really wanted to avoid it?"

And ChatGPT, being the sycophantic lie machine it is, rolled it's sleeves up and said "WELL, IN THAT CASE..."

And now Krafton look a bit silly.

https://www.theregister.com/2026/03/18/ ... k_founders

You may know Kraftron as the Korean publisher that owns Unknown Worlds, the development house behind Subnautica. If so, you're probably also familiar with the kerfuffle around the development and release of Subnautica II.

Early access was supposed to begin in 2025, but Kraftron delayed it, fired the company's founders, and seized control of Unknown Worlds in a bid to get out of paying the development house as much as $250 million if high earnings forecasts for the game turned out to be correct.

All of this was done at ChatGPT's advice.

According to a Delaware Chancery Court decision [PDF] this week, pretty much everything that Kraftron CEO Changhan Kim did at ChatGPT's urging in his bid to avoid that payout turned out to be a gross breach of contract.

Per the decision, ChatGPT told Kim that the earnout would be difficult to cancel, but Kim kept pushing the bot, asking it what steps to take anyway.

At ChatGPT's recommendation, Kim formed a task force with a mandate to either negotiate changes to the earnout or completely take over Unknown Worlds. ChatGPT advised that, were negotiations to fail (which they did), Kraftron should follow a specific sequence of events to ensure its success in the scheme, including preemptively controlling the public narrative by claiming that Subnautica II wasn't ready, and blaming the studio's founders.

ChatGPT also advised seizing control of distribution platforms like Steam to prevent Unknown Worlds from launching the game, and eventually firing the company's founding trio, with a made-up reason that they intended to release Subnautica II before it was ready, potentially damaging the franchise and harming earnings.

Kraftron followed the plan to a T. The Unknown Worlds founders sued for breach of contract.

During trial, Kraftron attempted to reframe the case, saying that the Unknown Worlds founders downloaded a bunch of data prior to being fired. They also argued that the founders asked to change roles to take on fewer day-to-day responsibilities, which was grounds to terminate their contract for violations of a business-as-usual clause. The judge laughed these arguments out of court.

In short, "none of Kraftron's proffered justifications have merit," Judge Lori Will said in her decision.

Fucking laughable. So many CEOs could just be binned, and replaced entirely with LLMs - even older, shitter LLMs. So many of them seem to be a liability, not a benefit, to the organisations they represent.

[Gavster]

It's mental that he'd rely so heavily on it, without sense checking via a legal professional. A friend of mine sent me a ChatGPT summary of how my legal situation with my neighbours should play out. I don't quite know why he did it, because I've got a fcuking solicitor who's been doing these kinds of cases for 20 years who gives me advice. Anyway, ChatGPT was 80% right, which is useful as a overview of the situation and nothing more.

[Rich B]

ChatGPT is useful for messing about with photos for draft visuals and filling blurb in presentation docs, but its legal abilities are more hassle than they’re worth ime.

One of our PDs thought it would be useful to help me (i was on AL that day) review a contract by sending over the AI analysis of the changes from a previous contract revision. it flagged up 13 changes, 2 of them serious. and he was going to send this back to the clients legal team.

when i compared the docs on adobe, the changes were either non-existent or were the revision numbers on the header. There were no actual issues. Essentially it would have made us look like dicks to our client.

[Jobbo]

It's mental that he'd rely so heavily on it, without sense checking via a legal professional. A friend of mine sent me a ChatGPT summary of how my legal situation with my neighbours should play out. I don't quite know why he did it, because I've got a fcuking solicitor who's been doing these kinds of cases for 20 years who gives me advice. Anyway, ChatGPT was 80% right, which is useful as a overview of the situation and nothing more.

Sorry, 80% right is not an overview of the situation - it is 20% wrong and therefore a terribly flawed view.

[DeskJockey]

https://nearzero.software/p/warranty-vo ... egenerated

Worth reading. Can see that play out in a not too distant future.

[Jobbo]

https://nearzero.software/p/warranty-vo ... egenerated

Worth reading. Can see that play out in a not too distant future.

Absolutely and I hope it doesn't come true. There must be enough resistance among normal human beings to prevent this becoming reality, surely.

One of the telling things is this:
“Your cabbage got worse because your specification doesn’t account for upstream model changes. It says ‘use weather data.’ It doesn’t say ‘alert me when the underlying weather models are recalibrated, because my crop maturity inferences are sensitive to the specific calibration.’ That’s a detail the AI has no way of knowing matters unless you tell it.”

People expect AI to know that detail, and far more that we don't even know ourselves, because it is intelligent. And it simply isn't - an AI model isn't fed with the experience of retiring farmers and the complete syllabus of an agricultural college, and even if both of those things existed online somewhere and were well labelled so could be found, it wouldn't know how much weight to attribute to them. AI instructions for things like this don't get direct feedback of the outcomes, they get a human interpretation of it so they don't necessarily learn fast or accurately.

AI is as much a philosophical discussion as a technical one. If AI could become sufficiently 'intelligent' that it is better than humans, we'll have created the thing which will usurp us.

[DeskJockey]

Fully agree. The TL;DR of the story is that prompt engineering will be a significant skill in a context where the underlying logic is invisible.

I'd much rather we didn't end up there though, as the more we abstract ourselves from the "floor" of any subject, the more vulnerable we become when it fails or, as in the story, changes without our knowledge and ability to fix.

[Beany]

Fully agree. The TL;DR of the story is that prompt engineering will be a significant skill in a context where the underlying logic is invisible.

I could make a career out of my experience playing the hitchhikers guide C64 text adventure game?

[DeskJockey]

Fully agree. The TL;DR of the story is that prompt engineering will be a significant skill in a context where the underlying logic is invisible.

I could make a career out of my experience playing the hitchhikers guide C64 text adventure game?

Once again Douglas Adams proves he was decades ahead of everyone else.