With the rapid development of cloud computing and Internet of things, how to store the explosively growing data becomes a challenge for storage systems. In tackling this challenge, approximate storage technology draws broad attention for its huge potential in saving the cost of storage and improving the system performance. Approximate storage techniques trade off the accuracy of the outputs for performance or energy efficiency taking advantages of the intrinsic tolerance to inaccuracies of many common applications. In this way, the applications improve their performance or energy efficiency while meeting the user requirements. Therefore, how to exploit the features of storages and fault-tolerant applications to improve data access performance, decrease space overhead, and reduce energy consumption is becoming a key problem for storage systems. In this paper, we first introduce the definition of approximate storage technology and show the techniques for identifying the approximate areas in the data. Then, we elaborate the approximate storage techniques for CPU cache, main memory, and secondary storage, respectively. We discuss the advantages and disadvantages of these approximate storage techniques along with the corresponding application scenarios. In the end of this paper, we summarize the features of approximate storage techniques and discuss the research directions of approximate storage techniques.