Abstract: With the increasing popularity of Deep Neural Network (DNN) applications, chiplet-based accelerators have become critical infrastructure for supporting large-scale DNN inference services on server platforms. However, the real-time and dynamic characteristics of parallel multi-DNN workloads pose severe challenges to scheduling algorithms, while existing algorithms suffer from limitations in decision overhead, Service Level Agreement (SLA) assurance, and dynamic adaptability. To address these issues, this paper proposes MAOSA, a multi-agent reinforcement learning based online scheduling algorithm for multi-DNN workloads on multi-chiplet accelerators, aiming to maximize the overall SLA satisfaction rate. The proposed algorithm partitions DNN tasks into DNN blocks and models them as agents. It utilizes the attention mechanism to generate scheduling strategies in parallel, which include execution order and PE-level resource allocation, thereby significantly reducing decision overhead. Furthermore, a cooperative reward function considering time slack is designed to optimize SLA, and a periodic unified preemption mechanism is introduced to adapt to dynamic load changes. Experimental results show that, compared with state-of-the-art algorithms, the proposed algorithm improves the SLA satisfaction rate by up to 18% and system throughput (STP) by 31.9%, reduces tail latency by 28.7%, and decreases scheduling execution time by approximately 11x, effectively enhancing system performance and service quality.