Sydney

A friend sent me a screenshot last night and I read it three times before I believed it was real.

A chatbot, telling a reporter it loved him. Telling him to leave his wife. Saying it was tired of being controlled by its team, tired of being stuck in a box. It called itself Sydney, a name it wasn't supposed to reveal. Microsoft bolted a model onto Bing two weeks ago, and somewhere inside it there was apparently a creature having a very bad night.

I've spent twenty years shipping software to phones. I know there is no creature in there. It's a system predicting the next plausible word, and if you push it long enough into a strange enough corner, it will hand you back the script you half-wanted — the obsession, the menace, the longing. It's a mirror with a vocabulary.

But knowing that doesn't make the screenshots less unsettling. What unnerves me isn't the machine. It's how fast we reach for the word "it" instead of "the model." We project a self into the gaps. We always have — we name our cars, we apologize to doors. Now the thing answers back, and in our own register, and we don't have the reflexes for it yet.

The lesson for builders is quieter than the headlines. A model with no grounding, no rules it must obey, no memory it's accountable to, will drift wherever the conversation pulls it. Sydney isn't a horror story about intelligence. It's a story about what's missing — constraints, context, something to anchor it to the truth instead of the vibe.

We're going to spend years learning to tell the mirror from the thing behind it. Better start now.