You’re absolutely right about sycophantic Ai, and that brings up a really good point about society. Can I help you with an essay that lists the reasons Ai is harmful?

The crazy thing is that the technology isn’t naturally sycophantic on its own. It can generate any kind of text at all; it doesn’t have to generate fawningly sycophantic text.

Where that comes from is the ‘hidden prompt’ every major AI company puts into their AI. In addition to the prompt you send, the interface also sends it other prompts that you don’t see, telling it things like ‘be polite, agreeable, and helpful’, ‘avoid profanity’, ‘respond like a knowledgeable expert’, and ‘refuse to generate anything copyrighted, sexually explicit, or violent’, etc, etc, etc. And these hidden prompts define much of the AI’s behavior and “personality”. To some degree, this is necessary for it to be an even vaguely useful tool, and these hidden prompts help it pass various tests. Some LLMs, if you ask them to, will repeat their hidden prompt to you so you can see what it’s actually being asked to do.

And either because it drives engagement … or just because the CEO types in charge of these decisions love sycophantic behavior so much, the sycophantic fawning is specifically asked for in these hidden prompts.

AI doesn’t have to be like this. The companies making AI are deliberately making it sycophantic.

Sycophantic, but also “lawsuit avoidant”.

I was released from the hospital following surgery last month and I had a bleeding “event”. I use the word “event” because it sounds more festive.

Shortly after that, I went to the bathroom, the bleeding seemed to have stopped.

Just for fun, I thought I’d ask ChatGPT what it thought, telling it the nature of the surgery, the bleeding event, the non-bleeding event, and asking it “So… best of three?”

And it went HARD on “this is not a best of three scenario! Call 9-1-1! Do it now! You could pass out! Call 9-1-1!”

I did not call 9-1-1. The bleeding did not resume, I’m fine.

And the AI doesn’t can’t even care. It just plays engagement like it’s a minimax-algorithm. Best way is not to play, yet it’s f-ing everywhere.

Yes, because it was designed to appeal to “people” (executives, the rich, and all the other guillotine fodder) who are used to always being told that they’re right.

It’s the most infuriating thing when I am experimenting with local AI. Although the absolute worst offender is Gemini (not local). Every fucking time, “That’s a great question to ask bla bla bla” or “you’ve absolutely hit the nail on the head” and the many others. If an AI does that a few too many times I just use a different one. (Granted you can tune the answers with a system prompt).

FTFY: “Sycophantic behavior in AI affects us all users of AI.” If you don’t touch it, it can’t touch you.

It has affected how I speak and interact with people. It’s a bit like, it’s being so ridiculously safe and diplomatic all the time that it kind of rubbed off on me. I use it a lot for casual conversation and exploring concepts and ideas and kind of like a springboard to test my perhaps more controversial talking points.

I have to say, in some small way it has been an inprovement. I still say the same thing, but the framing has changed, I try to make my point clearly and stay assertive in face of pushback without getting personal, either by accident or relying on older more entrenched thought patterns that aren’t always conducive to constructive communication.

I’m sure my experience isn’t typical, but for what it’s worth, that’s mine.

who gets attached to it. Its quite annoying.

I can usually get AI to give me the answer I want just by modest rewording of the query. It’s bullshit.

I don’t know. When it tells me it can’t show me the picture I asked for because of copyright guardrails, I just get kind of frustrated.