Quantifying image quality through subjective evaluation is very critical to image quality evaluation. Using the image quality ruler method, an average score per stimulus can be easily obtained in the unit of Just Noticeable Differences (JNDs). However, it requires a large number of subjects, since pure averaging does not consider the different judging quality of different subjects. In this paper, we propose an image quality evaluation framework using the image quality ruler method with a statistical model. By incorporating this model, we consider the quality score, the expertise of the subjects, and the difficulty of image rating task as three hidden variables. Then we use expectation-maximization (EM) to estimate these hidden variables. From our experimental results, we show that our method provides reliable results without using a large number of subjects. Preliminary results also demonstrate that the estimates of the parameters can guide us to better distribute the valuable human resources used to conduct psychophysical experiments.