People's attitudes toward mathematics are multifaceted. Across four studies, we found that children and adults have different attitudes about mathematics when asked specifically about whole numbers, as opposed to fractions. The vast majority of children and adults reported negative attitudes toward fractions despite having positive attitudes toward whole numbers. Across both children and adults, the difference in fraction and whole-number attitudes was present across levels of math achievement, indicating that it was not just participants who were worse at math whose attitudes differed by number type. These findings may have important implications for how children and adults engage with numerical information when presented as fractions.