Evaluation Resource Centre

Self-report recording

Self-report recording might be seen as including Diaries but a distinction is being made here because they tend to be approached a little differently.

A self-report, in this context, is where you ask the subject to report on what they have experienced in a particular situation.

An Example:

You modify your subject Web page and you are interested in the student response to it. You can use a self-report method by asking the students at the end of the first session to write down all their immediate reactions to this new page. You tell them that they can mention whatever they think is significant, both as positive and negative comment.

Type of Evaluation

The self- report method of collecting data is a qualitative method in that it normally uses an unstructured open-ended response format where the respondent is able to give his/her own version of the events.

Analysis of Data

The data you collect will have to be organised in ways that allow you to make some sense of what the respondents have said. There are various ways that you can do this.

Basic

Read each response and place it into a positive, negative and undecided pile.

Read each of these piles to see if there are any common themes within a pile.

Cross-check for positive and negative versions of the same theme.

More Advanced

Carry out a content analysis on all the responses.

Allocate each of the responses to the dominant area of content that it contains - as derived from the content analysis.

Compare the groupings of respondents that this creates - see if they have similar areas of content in the minor categories from the analysis.

Continue to reallocate respondents to possible descriptive groups until you think that you have a clear feel for what the respondents are telling you.

Reporting your results

Self-report data is usually reported in a descriptive form. From the analysis you can talk about the type of comments that have been made.

'Students tended to classify the Web site into very easy to use or easy to use. Very few suggested that it had any real problems.'

You can partially quantify your data if that is your preference. So, from the positive, negative and undecided groups you might be reporting:

'The basic classification of the data into basically positive or negative response as well as those who are undecided, showed the following pattern:

Positive 62%
Negative 15%
Undecided 20%

The above figures do not add to 100% because there was a small number (3 responses) that could not be classified because their responses were incomprehensible.'

When reporting raw figures and percentages beware of making too much of differences you have noted. From the above example we could say:

This classification of the data clearly shows that the majority of the students were positive about the changes. Changes were made to the Web site as a result of negative responses in the previous semester. We feel that the level of positive response shows that the changes have been effective.

But, if the results were more like:

Positive 48%
Negative 15%
Undecided 35%

You would be hard-pressed to say that the is a clear positive response. More likely you would be looking at the content of the undecided comments to see if that could help in the next stage of Web development.