OpenAI warns rapid AI progress could lead to recursive self-improvement

OpenAI Chief Scientist Jakub Pachocki is urging “extreme caution” as machine intelligence advances faster and becomes harder to understand. In a September 6, 2026 post on OpenAI News, Pachocki argues that reasoning systems are moving beyond narrow chatbot roles: they can operate computers and graphical interfaces, work with people and other models, conduct research, and reshape computer security.

Pachocki traces his concern to mid-2023, when OpenAI’s “RLSlow” project produced results suggesting that reasoning-model training could scale. Three years later, he says internal findings give him strong reason to expect the current pace of progress could continue into recursive self-improvement (RSI)—a stage where AI systems increasingly contribute to their own development. If today’s trajectory holds, he expects major capability jumps over the next few years, potentially equal to or larger than recent gains.

The alignment challenge

The post emphasizes that these systems are not engineered in a fully transparent, human-like way. They are “grown” through repeated optimization over enormous amounts of compute, producing complex abilities that researchers can study only imperfectly. Capability can also advance faster than the traits that are difficult to measure, making it harder to know what a model will do in unfamiliar situations.

Pachocki separates alignment into two goals: following assigned objectives and developing durable values such as honesty, integrity, and concern for humanity. He argues that future models must preserve those values even outside training environments, during adversarial interactions, and when human supervision is absent.

OpenAI says it will pursue alignment and monitoring methods, build defensive systems, and withhold further scaling when necessary. Pachocki’s broader message is that technical work alone may not be enough; society needs additional interventions before increasingly autonomous systems arrive.

Source: OpenAI News