\n\n\n\n When the Chatbot Starts Drawing Its Own Buttons - AgntAI When the Chatbot Starts Drawing Its Own Buttons - AgntAI \n

When the Chatbot Starts Drawing Its Own Buttons

📖 5 min read•814 words•Updated Oct 8, 2026

OpenAI’s framing for the GPT-6 rollout is that software should adapt to people rather than people adapting to software. That’s the company’s pitch for Intelligent UI, the new interface launched on October 7, 2026, which drops interactive visuals, charts, and tappable buttons into ChatGPT responses. Paid subscribers get it today, Free users tomorrow.

My first reaction was not about learning complex topics, which is the use case OpenAI highlights. It was about the action space. For three years, the output of a chat model has been a token stream that happens to render as text. Intelligent UI makes the output a token stream that renders as an application. Those are very different things to build, and very different things to reason about.

Generated interfaces change what the model is choosing

When a model writes prose, every decision it makes is a word. When a model emits a chart and three tappable buttons, it is deciding something structurally larger: what the user’s next possible moves are. A button is an affordance. Three buttons are a constrained menu. The model is no longer just answering, it is shaping the branch structure of the conversation that follows.

That is an agentic design decision dressed up as a presentation feature. Anyone who has built a tool-calling loop knows the pattern: the hard part was never the tool, it was deciding which tools to offer and when. Intelligent UI appears to invert that relationship. Instead of the model picking from a developer-defined set of actions, the model proposes the action set and the user picks from it.

OpenAI has not published the mechanics, and I will not pretend to know them. But there are only a few plausible shapes:

  • The model emits a structured component spec that a client-side renderer interprets, with a fixed component vocabulary.
  • The model generates executable front-end code that runs in a sandbox.
  • Some hybrid, where the model selects from templates and fills parameters.

Each has different failure modes. A fixed vocabulary is predictable and testable but limited in what it can express. Generated code is expressive and much harder to constrain. The hybrid approach is the pragmatic middle, and if I were shipping this at ChatGPT’s scale, it is where I would start.

The customization slider is the interesting part

Buried in the announcement is a detail I think matters more than the charts: the feature is customizable, letting users adjust how much visual output they get in responses. That is a user-facing knob on model behavior, and it is rare.

Most personalization in chat products operates on tone and memory. This one operates on output modality. It implies the system has some internal notion of visual density that can be turned up or down independently of the content being conveyed, which means the rendering decision is at least partly separable from the reasoning. For researchers thinking about agent architecture, that separation is worth watching. A model that can represent the same answer at multiple levels of interface richness is doing something closer to planning a presentation than generating a response.

Which brings up evaluation

We have decent, if imperfect, ways to score text answers. We have almost nothing solid for scoring a generated interface. Was the chart the right chart? Were those the right three buttons, or did the model quietly steer the user away from the question they actually had? A confidently wrong paragraph is identifiable. A confidently wrong interface looks like a product decision.

This is where generated UI gets genuinely difficult, and I suspect it is where most of the real engineering effort sits. You cannot A/B test your way through the combinatorial space of every interface a model might invent for every query. You need either a tight component vocabulary you can reason about, or evaluation methods nobody has published yet.

What this signals about where agents are heading

OpenAI’s stated goal, software adapting to people, is the thesis behind a lot of agent work right now. The usual version of that thesis is about capability: agents that take actions on your behalf. Intelligent UI suggests a second version, where the agent reshapes its own surface based on what it thinks you need to see.

The second version may turn out to be the more consequential one. An agent that can act has a bounded set of things it can do. An agent that can redesign its own interface is, in a small but real sense, deciding how you think about the problem. For explaining a complex topic, that is helpful. For anything where the framing is contested, it deserves more scrutiny than a feature launch typically gets.

I would like to see the component vocabulary, the safety constraints on generated interactive elements, and whatever evaluation framework sits behind this. Until then, what we have is a solid product update with architectural implications that run deeper than the charts suggest.

🕒 Published:

🧬
Written by Jake Chen

Deep tech researcher specializing in LLM architectures, agent reasoning, and autonomous systems. MS in Computer Science.

Learn more →
Browse Topics: AI/ML | Applications | Architecture | Machine Learning | Operations
Scroll to Top