Reading results
Every analysis gives the same four parts: a one-sentence answer, assumption checks, a chart, and the full tables.
The answer
The first sentence says what the data show and the test behind it, for example:
Diet affected ADG_kg, after allowing for Barn (F(3, 9) = 19.74, p < 0.001).
Read it as: the diets differed in average daily gain, and differences this large would be very unlikely if the diets truly had the same effect. The numbers in brackets are what a paper reports: the test statistic, its degrees of freedom, and the p-value.
When the answer says "No effect … was detected", the data did not show a difference. That is not proof of no difference: a small study can miss a real one. See power.
p-values and the confidence level
- A p-value is the chance of results at least this extreme if there were truly no effect. aStat calls a result significant when p is below 1 minus the confidence level: below 0.05 at the usual 95% level.
- Values below 0.001 are shown as
p < 0.001. - You can change the confidence level to 90% or 99% in the setup.
Confidence intervals
A 95% confidence interval is the range of values the data are consistent with. A difference of 12 g/day with an interval of 4 to 20 means the true difference is plausibly anywhere from 4 to 20. An interval that includes 0 (or 1, for ratios) means no clear difference. Intervals tell you more than p-values: how large the effect might be, not just whether it exists.
Assumption checks
Each analysis checks what its method relies on and labels each check:
- OK: the check passed.
- Check: worth a look; the message says what to consider, such as a transformation or the rank-based alternative aStat already ran.
- Problem: the results may not be trustworthy, for example a group with fewer than 3 values.
Common checks: normality of residuals (Shapiro–Wilk), equal spread across groups (a Levene-type test), sample size, overdispersion for counts, and variance components estimated as zero in mixed models (the model is then simpler than planned, but still valid).
Means: raw and adjusted
Model-based means (LS means, called estimated marginal means in R) are adjusted for the other terms in the model: blocks, covariates, or the other factor. They are the means SAS's LSMEANS statement reports, and the ones to put in a paper's tables. With missing data or incomplete blocks they can differ from the raw averages; the adjusted ones are the fair comparison.
Comparing pairs (Tukey)
When three or more groups differ, aStat compares every pair with Tukey's adjustment, which keeps the chance of any false difference across all pairs at 5%. Each pair has a difference (or a ratio, for yes/no and count models), its confidence interval, and an adjusted p-value.
Degrees of freedom
Mixed models report fractional denominator degrees of freedom, such as F(2, 17.6). These come from the Satterthwaite or Kenward–Roger methods, which SAS also uses, and give honest p-values when there is more than one source of random variation.
Effect sizes
Where it helps, aStat gives a standardized effect size with Cohen's labels (small, medium, large): Cohen's d for t-tests, eta squared for one-way ANOVA. These help compare across studies, but a "small" effect on feed efficiency can still be worth a lot of money; judge by the units too.