Abstract: Based on the factored representation of a state, a new SARSA（ λ ） algorithm is proposed. The main principle of the algorithm is that a heuristics on the state similarities can be gained from the features of the state, and according to the heuristics, the state space is aggregated, significantly reducing the searching and computing complexity for the state space. Therefore the algorithm is a promise for solving large scale MDPs problems which are of a huge state space.