Usually, people pursue the accuracy of data in scientific research, especially the sensitivity of indicators. If anyone is against this pursuit, it is estimated that everyone will rise up against him. It is expected that P value, which is currently at the center of controversy in statistical significance test, is such an accurate and sensitive indicator. To abandon or forbid such an indicator, most people will not agree with it by intuition. However, the criticism of P value is that it is too accurate and sensitive - especially, it is related to the sample size, which can be used by researchers to obtain significant results by expanding the sample size.

In order to solve the problem that P value is too sensitive, one scheme is to use the effect quantity to measure the influence of one variable on another. According to different statistical tests, there are different effect quantity indicators, commonly used are Cohen's D and D η 2（Cohen, 1988）。 Where d = 2T / √ DF (DF squared), t is the value of t test, DF is the degree of freedom; η 2 = fdfb / (fdfb + DFW), f is the value of F test, DFB and DFW are the degrees of freedom between groups and within groups respectively - generally, DFW is the degree of freedom of error DFE (in short, DFB and DFW are the molecular degrees of freedom and denominator degrees of freedom of F test).

In the general statistical significance test report, the sample size, mean m and SD, t value or F value, DF and P value of each group (Group) will be given. According to this, the statistic D or D of the effect can be calculated η 2。

Those who think it is unnecessary to use p value are based on the above facts, that is, the effect quantity can be calculated from the results of statistical significance test. Therefore, it is unnecessary to report the effect quantity and P value. In fact, other statistics can be calculated from some statistics, and there is not enough reason to show these statistics at the same time. For example, the degree of freedom can be calculated by group and sample size; From the mean value, standard deviation and degree of freedom, we can calculate the standard error and the statistics of common statistical tests (for example, t value or F value). However, no one thinks that it is not necessary to present the statistics of group, sample size, mean value and standard deviation.

The following is a case study to illustrate the relationship between p-value and effect quantity (in Chinese) η 2) and the differences between them.

The researchers investigated the influence of black-and-white background or gray background on moral judgment. The experimental materials were six social problems (pornography, adultery, drug use, littering, smoking and profanity). They were asked to use a scale of - 5 (= very immoral) to + 5 (= very moral) to evaluate their morality. Researchers predict that priming by black-and-white visual contrast will lead to more extreme moral judgments than none (zarkadi & schnall, 2013).

In order to evaluate the polarization of moral judgment, the researchers calculated an index of deviation score, that is, the distance between the judgment result and the midpoint of the scale. The results show that the mean value of deviation score in black and white condition (M = 2.50, SD =. 96) is larger than that in gray condition (M = 2.05, SD =. 91), f (1128) = 7.35, P =. 008, η 2 = .05。 When the researchers analyzed the six subjects separately, they showed the same general pattern, in which smoking (f (1128) = 5.69, P =. 02, P =, η 04), drug use (f (1128) = 4. 31, P =. 04, η 03), adultery (f (1128) = 8. 34, P =. 005, η The main effect of start-up condition was significant. In addition, there was no significant difference in the mean severity between the two conditions (black and white condition: M = -1.79, SD = 1.57; black and white condition: M = -; Grey condition: M = - 1.05, SD = 1.32), f (1128) = 1.05, P =. 31.

As mentioned above, the effect quantity can be calculated manually from the statistical significance test results reported in this paper（ η 2）。 Here are the corresponding situations:

(1) F (1128) = 7.35, P =. 008, η 2 =. 0543, keep two decimal places, i.e η 2 =. 05, the same as the data reported in this paper.

(2) F (1128) = 5.69, P =. 02, η 2 =. 0426, i.e η 2 =. 04 (as in the text).

(3) F (1128) = 4.31, P =. 04, η 2 =. 0326, i.e η 2 =. 03 (as in the text).

(4) F (1128) = 8.34, P =. 005, η 2 =. 0612, i.e η 2 =. 06 (as in the text).

(5) F (1128) = 1.05, P =. 31, η 2 =. 0081, i.e η 2 =. 01 (not presented).

The above p value and effect quantity were compared η 2, it is easy to find that P value is larger than the effect quantity η It's much more sensitive. For the same F test (i.e. DFB and DFW are the same), when p =. 31, P =. 04, P =. 02, P =. 008, P =. 005, the corresponding η 2 = .01、 η 2 = .03、 η 2 = .04、 η 2 = .05、 η 2 = .06。 In particular, when p value changes from 0.31 to 0.04, there is a significant difference η 2 only changed from. 01 to. 03, not obvious; When p value changes from significant 0.02 to extremely significant 0.005, η 2 only changed from. 04 to. 06.

Combined with the experimental materials, the p value and the effect quantity can also be clearly defined η The two indicators can complement each other. For example, the background of black-and-white contrast can polarize moral judgment, which is related to specific social problems, and has an impact on adultery judgment (P =. 005, η 06) vs. smoking judgment (P =. 02, P = 0, η 04) is more obvious - at this point, the sensitive p value is impressive. Especially, the effect of black-and-white background on the severity of moral judgment is very small（ η Therefore, researchers can understand that there is no need to examine the impact of black-and-white contrast on moral judgment. Even if the sample size is expanded and the effectiveness is improved, it is of little significance.

When used η Cohen (1988) used 0.10, 0.30 and 0.50 as the criterion of small, medium and large effect respectively. Therefore, the case study shows that the effect of black-and-white contrast on the polarization of moral judgment does not reach the standard of small effect. This shows that the influence of black-and-white contrast on moral judgment is quite subtle, and it may be hard to detect - although the influence is small, it does exist. Then, for other researchers, if they want to repeat this study, they need to pay attention to improve the effectiveness (for example, increase the sample size), otherwise, they may not get significant results. This further suggests that psychological research reports both p-value and effect quantity, which is not superfluous, but complement each other.