What is Reliability and Validity?

                 

In statistics, there are many elements to consider if someone wants to write questionnaires to obtain high-quality data. Researchers want to make sure that their measures can both be applied consistently as well as accurately reflect the objectives they seek to measure. Therefore, they strive to obtain a high amount of reliability and validity in their studies. 

What does that really mean and why does it matter?      

Validity is concerned with the accuracy of the researcher's measurements. This is often discussed in the context of sample representativeness but it is also affected by the preciseness of the questions that measure what it's supposed to be measuring. 

Researchers strive for high validity in two forms: internal and external validity. Internal validity looks into whether the questions being asked can appropriately measure the outcome that the researchers intend to study. Meanwhile, external validity refers to the extent by which the results can be generalized to the target population the survey sample is representing     

Reliability, on the other hand, is concerned with the consistency of the measure or the degree to which the questions used in the questionnaire elicit the same type of information each time they are used under the same conditions. For example, a measure is reliable if it yields consistent scores across multiple administrations.

Similarly, there are two types of reliability: internal and external reliability. Internal reliability assesses the consistency across items on the test- the extent to which a measure is consistent within itself. External reliability refers to the extent to which a measure varies from one use to another. 

This can be measured using a variety of methods, such as internal consistency, equivalent forms, and test-retest approaches. Internal reliability forms include an internal consistency estimate of reliability, in which individuals are administered a measure with multiple parts once. External reliability includes giving the same measure twice or a slightly different measure each time for two occasions.        

One helpful metaphor is that of a target. We can imagine that a study was conducted on a group of people, in which each person is represented as a single shot at the target. If the measure lacks validity but has reliability then it is consistent but wrong. Likewise, if a measure has validity but lacks reliability then it is accurate but only in a singular case - and is therefore not consistent when applied in a general context.  

Now you might be wondering how I took both of these in consideration in my study.

Earlier, I came across a study by Larsen and Leroux (1982) which stated that item analysis tests, compared to factor analysis tests, proved to be superior for reliability and validity estimates.

As a result, when I first did my factor analysis two weeks ago, I took into account the importance of reliability and validity and computed item analyses for all my variable scales. This meant I could get an accurate measure of my questionnaire's internal consistency providing me with good results.

Overall, it is essential for researchers to strive for a high level of validity and reliability so that they can collect objective and high-quality data while fulfilling their goals in conducting the study.

Semajno Naŭ (9)

Hello again fascinated - and fascinating - psychology readers! This week I did quite a few statistical analysis tests in order to derive some meaning from all the numbers.

At first, I did a factor analysis on the items of conformity and derived factors that had statistical significance present in the study.

In statistics, underlying correlations between certain items that have a role in the data are called factors. For example, factor one for conformity might indicate being persuaded by others while factor two is the desire to match one's standards to others. All of these factors are a specific indication of something greater happening "behind the scenes" of the numbers and reflect some aspect of conformity.

Here is a picture of the factor rotation that I did when I was looking into potential factors that existed.

This indicates that there are only three factors with an Eigenvalue greater than one. As a result, I checked the component matrix to see which items were associated most strongly with factors 1, 2, and 3.

After that, I did item analyses to see which items correlated most strongly within the factors. In statistics, items are the formal definition of the questions/statements that are answered in the survey by the participants.

The most exciting outcome of these tests was that I was able to narrow down the original 8-item scale to a 4-item scale with two distinct factors that controlled for over 64% of the total variance! Sweet!

The important thing to note is that actually performing a statistical test is simple as the program computes the numbers for you. However, the challenging part is when you attempt to assess all the components of your study and figure out how you are to approach it statistically. Essentially, statistics is easy to do once you know what you are doing and why you are doing it.

Finally, I wanted to see how conformity related to everything else so I did correlation analyses between the factors and all the other variables. This meant having TONS of correlation tests between the conformity factors and each of the other variables (there about 12 sets of variables).

There is much to describe here but its more technical and numerical than what can easily be understood. But I promise it will all be incorporated in my final research paper.

As always, stay motivated and see you next time!

Septem Octo

Hello, Everyone! 

The exhilaration is continuing as we break the barriers into week number eight! We are two-thirds of the way to the finish line! This week I had to purchase the exclusive and elusive statistics computing software package SPSS 16. 

Here is an image that pretty much summarizes my week and progress on the senior project. 

I had to spend approximately seven hours to input the 50 questions/variables from 175 surveys, but the fun does not stop there. Now that all the numbers are inputted, I have to spend time figuring out which statistical analysis tests to use. Next, I will have to create descriptive statistics and then figures to represent the numerical data through graphs, tables, or charts. Finally, and most importantly, I will have to interpret the data and find some practical meaning to all this relating to my original research questions. 

Stay tuned, as I am ever so slightly inching my way towards unlocking the secrets behind motivation and beyond! 

Until next time!