Validation of competency-based questionnaire
Today it was a statistic’s day for me. I decided to verify our techniques and methods on their validity and reliability. So I started from the questionnaire on competencies elaborated for one of our educational programs “Functional Testing”.
Previously, I wrote that I had made a questionnaire to assess competencies of our students which are included in the model of competencies of software testing specialist. With its’ help we managed to evaluate competencies of more than 30 students.
First of all, I was interested in the reliability of the questionnaire, mainly its internal consistency. After making some research on this issue, I found a coefficient Cronbach’s alpha thanks to which by simple mathematical calculation you can measure internal consistency.
The result turned out to be more than pleasant. The Cronbach’s alpha coefficient is 0.98 which means that internal consistency of the questionnaire is very good and it proves that the questionnaire assess those competencies for evaluation of which it was created.
Now we can implement this questionnaire without hesitation to make self-evaluation.
Moreover, I verified also construct validity of the questionnaire by taking as an external criterion the results of examination paper of students as we suppose that having a high level of competencies included in the model of competencies of a software testing specialist, this person will be successful in fulfilling his duties at work. Taking into consideration that an examination paper consists of practical tasks which our student will meet at work, I decided to use a mark for an examination paper as a criterion of job effectiveness. This examination paper is prepared and checked by independent SMEs at the end of study to assess success of our students.
To find relationship between competencies and high marks for examination paper I used Spearman’s rank correlation coefficient (Spearman’s rho). Spearman’s rho was calculated for each competency and varies from 0.53 to 0.73 where the lowest correlation is for Initiative and the highest one for Responsibility. So, construct validity of the questionnaire is high enough.
Consequently, we can say with confidence that the more is the level of certain competencies of a testing specialist, the more high will his job performance.