Cognitive radio vehicular ad hoc networks (CR-VANETs) have been envisioned to solve the problem of spectrum scarcity and improved spectrum resource efficiency in vehicle-to-vehicle communication by exploiting cognitive radio into the vehicular ad hoc networks. Most existing routing protocols for cognitive radio networks or vehicular ad hoc networks cannot be applied to CR-VANETs directly due to the high-speed mobility of vehicles and dynamically changing availability of cognitive radio channels. At present, the routing research for CR-VANETs is relatively few. How to utilize the spectrum resources effectively and moreover reduce the spectrum band consumption caused by routing hops is still a pending problem. Aspiring to meet these demands and challenges, this paper presents a joint routing and scheduling, which combines the scheduling of spectrum resources and the goal of minimizing routing hops in CR-VANETs. To achieve this goal, we first establish a network model and a CR spectrum model to predict the contact duration between vehicles and the probability of spectrum availability. We define the communication link consumption and the weight of channel according to these parameters. Then we transform the optimization objective into a routing scheme with minimizing hop count, subject to constraint on the scheduling of spectrum resource, and moreover prove this routing scheme is NP-hard. To tackle this issue, a hybrid heuristic algorithm is composed by a particle swarm optimization with fast convergence and a genetic algorithm with population diversity. Simulation results demonstrate that our proposal provides better routing hop counts compared with other CR-VANETs protocols.