Towards Architecting LLM-based Systems

Large Language Models (LLMs) are increasingly integrated into complex software systems across multiple domains. This shift adds to system complexity and amplifies architectural challenges to ensure system-wide qualities such as scalability, reliability, and evolvability. Software architecture can play a central role in addressing these challenges by structuring systems and guiding quality-driven design decisions. Currently, however, it is not clear what the software architectures of LLM-based systems should be. The main contribution of this paper is to present a broad view on the software architectures of LLM-based systems in terms of architectural patterns and organization. To this end, we scrutinized the literature on these systems and found 99 architectures, and found that most do not have architectural design tasks as a primary concern. Additionally, a few approaches—two architectural reasoning taxonomies, one reference architecture, and three domain-agnostic architectures—for supporting the architectural design have been proposed to date. Therefore, the research community and practitioners should join efforts to leverage the field of LLM-based architecture with, for instance, architectural design and evaluation methods, to ensure the qualities of these systems (e.g., safety and reliability), as they have increasingly performed critical tasks.