Rating scales are increasingly used to perform patient assessments through graded responses on ordinal level of measurements. Outcomes measured in rating scales however provide only a rank ordering of scores where differences between scores composing the scale are unequal in terms of what is being measured. Scores from rating scales do not have an underlying number line with an equal interval and are unable to represent a true linear and continuous measurement. As such, they may not be able to comply with many assumptions of parametric statistics, and has the potential to produce invalid statistical inferences. This paper highlights current concerns in using total raw scores in measuring outcomes in clinical settings along with their potential solutions through using Item Response Theory (IRT) modelling. Results from a simulation study and other real life examples are examined to investigate the suitability of using parametric statistics in the rating scale scores (e.g. summative scores) and their IRT transformed scores. The findings support the use of IRT-based scores when using rating scales to measure patient progress, which in turn can help clinical decision making to optimize patient care.