Individual Cognition and Complexity Limits单体认知与复杂度上限

The following content is generated by LLMs and may contain inaccuracies.

Context

This note addresses an old problem at the intersection of cognitive science, organization theory, innovation economics, and software engineering, one that has become more urgent with the emergence of AI agents. The original text makes four claims:

1. The world is simply there. It has always been there; the limitation lies with the observer.
2. There is a ceiling on the complexity a single human can handle. Once things exceed a certain level of complexity, additional people are needed to take on the work and to design the next level of complexity and refinement.
3. Some things remain undiscovered, partly because of insufficient demand and attention. For complexity to persist, someone must continually care about it and maintain it.
4. The author is hitting a wall. While building Latere AI, complexity has kept rising. Even with agent assistance, the author finds it increasingly difficult to further understand and manage the expanding complexity, and the whole project has gradually stalled. The author therefore argues that “the spontaneous, autonomous, and self-evolving nature of system design is urgent.”

This tension is especially sharp now. Agents have greatly reduced the cost of “producing complexity”: code, features, and branches arrive quickly. But the cost of “understanding complexity” has barely fallen. The gap between production and understanding keeps widening, and the cognitive bandwidth of the solo developer has become the bottleneck for the entire system.

---

Key Insights

1. The “single-person complexity ceiling” has quantitative evidence in innovation economics

• Benjamin F. Jones proposed the mechanism of the “burden of knowledge”: as knowledge accumulates, later innovators have more to learn, so they must narrow their expertise and rely increasingly on teamwork, which also drags on long-run growth. Using micro-level data on inventors, he tested six predictions of the model. The paper’s title is directly “The Death of the Renaissance Man” (Jones, The Burden of Knowledge and the ‘Death of the Renaissance Man’, NBER w11360). This is nearly the same proposition as the original text’s claim that “additional people are needed to take on the next level.”
• Bloom, Jones, Van Reenen, and Webb found that research inputs have risen sharply while research productivity has fallen dramatically. Today, the number of researchers required to double chip density is more than 18 times the number needed in the early 1970s (Bloom et al., Are Ideas Getting Harder to Find?, AER 2020). This indirectly supports point 3 of the original text: deeper levels of complexity require multiplied attention to “sustain” them; otherwise, they will not be explored.

2. Adding people does not scale the complexity ceiling linearly: Conway’s warning

• Melvin Conway’s 1968 paper describes almost exactly the author’s current situation: when the apparent complexity of a system approaches the designer’s limit of understanding, the designer tends to want to delegate the task. This is the turning point of design: “Either he struggles to simplify the system until it is comprehensible and wins, or he loses control of it” (Conway, How Do Committees Invent?).
• The same paper also notes that the number of possible communication paths in an organization is roughly half the square of the number of people; each delegation narrows the design space that can be pursued; and system structure replicates the communication structure of the organization, which is Conway’s Law. Therefore, “additional people” themselves introduce new complexity. The original text’s phrase “designing the next level” needs one more clause: the structure of the next level will be shaped by the organizational form of those who take it on.
• Extending this to the agent era: if we regard agents as “additional people,” Conway’s Law suggests that system architecture will converge toward the communication topology between humans and agents. The author’s feeling that “even with agents, it can’t be handled” may stem not only from insufficient agent capability, but also from the fact that the interface between human and agent (context, memory, transmission of intent) is itself as narrow as a straw.

3. “Conservation of familiarity” in software evolution: stagnation is a law, not personal failure

• In Lehman’s laws of software evolution, the second law, “continuing growth,” states that the complexity of E-type systems will keep increasing unless work is specifically invested to maintain or reduce it. The fifth law, “conservation of familiarity,” states that all stakeholders must maintain their mastery of the system’s content; excessive growth weakens this mastery, so the average incremental growth remains constant. The eighth law points out that the evolutionary process is a “multi-level, multi-loop, multi-agent feedback system” (Lehman’s laws of software evolution).
• This provides a structural explanation for the “slow stagnation” described in the original text: the growth rate is constrained by the “familiarity” of the maintainers. Agents have increased the rate of writing code, but not familiarity. Therefore, stagnation is predictable, and the remedy lies not in “writing faster,” but in lowering the cost of maintaining familiarity, or in making the system no longer require humans to be familiar with it.

4. Self-evolving systems: existing feasible paths and their costs

• The Darwin Gödel Machine (DGM) is currently the empirical work closest to the original text’s vision of “spontaneous, autonomous, self-evolution.” The system repeatedly modifies its own code and uses benchmark testing for empirical validation instead of impractical formal proofs, while maintaining an open-ended archive of agents as “stepping stones.” The results: SWE-bench performance rose from 20.0% to 50.0%, and Polyglot rose from 14.2% to 30.7% (Zhang et al., Darwin Gödel Machine, arXiv:2505.22954).
• Three direct implications for Latere AI arise from the same paper:
  • Open-ended exploration matters more than hill-climbing. The baseline that removed the archive and modified only the latest version stagnated quickly, and the lineage of the final best agent even went through two performance drops.
  • The evaluation function is the direction of evolution. The premise of DGM is that “an increase in benchmark scores equals an increase in self-improvement capability.” Without reliable, automated evaluation signals, self-evolution is out of the question.
  • Self-evolution intensifies incomprehensibility. The authors themselves acknowledge that iterative self-modification may produce increasingly complex and inexplicable internal logic, hindering human understanding and oversight. This directly conflicts with the motivation of the original text: the author wants the system to take on complexity in their place, at the cost of further decline in the author’s understanding of the system.
• Also note that the exploration process of DGM itself (archive maintenance, parent selection) remains fixed and manually designed. Even in the most advanced work, the “meta-level” still requires humans to design it; the ceiling of complexity has merely been raised by one level.

5. Counterarguments and supplementary perspectives

• Hierarchical decomposition is humanity’s original solution. In The Architecture of Complexity (1962), Herbert Simon argued that complex systems tend to evolve into “nearly decomposable” hierarchical structures, because stable intermediate forms allow evolution to proceed faster. Following this line of thought, the “next level” in the original text does not necessarily require more people; it can rely instead on better abstraction boundaries, so that each level is below the ceiling for the cognizer (no online link available here).
• The Law of Requisite Variety. Ashby’s Law of Requisite Variety states that the variety of the controller must be no less than the variety of the system being controlled. A single human cognizer facing an expanding system will inevitably lose control. This law also implies that self-evolving systems internally need sufficient “regulatory variety”; otherwise, the loss of control is merely shifted from humans to the system’s interior (no online link available here).
• “The world is simply there” is a realist position. It can be contrasted with another view: much complexity is humanly constructed, existing because it is needed rather than discovered. This precisely echoes point 3 of the original text: “insufficient demand and attention” can both cause existing things to remain undiscovered and cause things that could have been built to never be made at all.

---

Open Questions

1. Would self-evolution merely replace “understanding” with “trust”? If Latere AI can evolve autonomously, the author’s relationship to the system shifts from “understanding it” to “evaluating it.” Then the real bottleneck may lie not in whether the system can evolve, but in whether the author can design an evaluation function they still understand, and then use it to constrain a system they do not understand. Is the complexity of the evaluation function also subject to a single-person ceiling?
2. Can attention be generated by the system itself? The original text says that some things go undiscovered because of a lack of demand and attention. Open-ended exploration (novelty search, stepping-stone archives) is essentially using algorithms to manufacture “artificial curiosity.” For Latere AI, rather than pursuing “self-optimization,” should it first build “self-attention,” allowing the system to judge for itself which complexity is worth maintaining and which should be forgotten?