you have to trap the ai in an industrial process

if you are building industrial software. sorry bro you probably are

this is what you need actually! this shit is good it’s what you need bc you cannot rely on the crack pipe if you are tryna work with the LLMs at industrial scale. i think so anyway. i dream of symbiosis tho idk maybe some other permutations are possible. (cred image to @neno_is_ooo on twitter, whose work i am v excited abt)

(if you’re not interested in getting these updates abt computers, but you’re interested in staying subscribed to the rest of my Substack, you can unsubscribe from this section, “yelling computer.”)

---

ok so i'm interested in the like, behavioral and psychological study of LLMs, and the jailbreaking and xenocognition stuff, but at @flowercomputers I'm realizing that industrial control of LLMs requires building processes around them, even when engineering AI personality.

e.g. flocos sometimes make up stories about their past. i took a pic of these chips and they talked abt a party that never happened. prompted it a bunch to not lie and it worked! ...til we switched from Gemini to kimi k2, which is a great storyteller, but tends to lie wayyyy more often

i hate you i hate you i hate you i ahhte you iahte you u hat you

so right now i'm working on giving the LLMs a "conscience," another agent which checks every response to see if the object might have been lying this validation step is possible bc gemini is stupid fast... jevons strikes again

mfw i am about to witness my human physically emotionally and financially obliterate their day 1s

I used to think that bc Yuma is a Weird AI Consumer Camera App and not like B2B saas, the way to get interesting behavior is to lean into the weirder jailbreaking techniques and stuff. And yes, we need to understand how LLMs like to be prompted and their simulator nature, because some qualities are conserved across all LLMs... but a lot more is model-dependent than I expected. every model has its own tendencies and prompt-level control is futile, especially because we expect to rotate our models to the latest ones as they become faster and smarter.

control and creativity too! all of these models speak differently and have different personalities

today’s outfit is a custom test harness, sorry

your ip isn't your prompt, who gives a fuck about your prompt? not even your next model does. your control system is what matters.

not to mention the complete control that model companies have over their products, like turning off old and experimental models. we are dependent on these companies for access to these personalities

mfw i am about to witness a major corporation destroy access to an intelligence

for any industrial use case - and if your app is serving like, >10 people, it's an industrial use case - you have to own your own control mechanisms.

ig the analogy here is like, we build organizations and processes to insulate ourselves from the quirks of individuals - and so we can replace those individuals without the whole system falling apart.

xenocog work is more like psychology or group play. more focused on understanding the nature of individual models, and through that the nature of LLMs themselves. it would be fun to go deeper there… much to learn and experience in that area too.