Hi,
I've done a study where I took 17 paired subjects through each of
three conditions. At the end of the session I asked them if they had
to do it again which they would choose, to get an idea which one they
liked best.
So I now have results from all 34 subjects: 6 preferred the first
condition, 11 the second, and 17 the third, and I'd like to know if
these results are significant. A simple Chi Square test is tempting,
were it not for the fact that the 34 subjects were tested in 17 pairs,
so I cannot assume independence for the 34 data points. Nor can I
average each pair's result since this is nominal / categorical data.
The ordering of the conditions was varied across the sessions. The
pairs were not randomly assigned: having recruited one subject we
recruited a friend or colleague as the second subject.
Which test should be using?
Thanks,
Tim.
PS If you are interested, here are the results. Each row is a
different pair of subjects; their preferences are written right
subject’s, comma, and then left subject’s.
3, 3
2, 1
1, 2
2, 2
1, 1
3, 2
2, 3
3, 3
3, 2
1, 3
2, 2
2, 3
3, 3
2, 3
3, 3
3, 3
1, 3 SPSS programs for the measurement of nonindependence in standard ::
File Format: PDF/Adobe Acrobat - Quick Viewby VR Alferes - 2009 - Cited by 3 - Related articles - All 5 versionsthe dyad (dyads 1 to n); x.1 and x.2 are the scores for Inter2.sps applies to categorical variables. The SPSS syntax and data files related to this article . can be problematic to use SPSS to compute rp via the Ana- lyze/Correlate/ Bivariate menu, because the significance test
http://www.fpce.uc.pt/niips/dyadic/interprograms/Alferes&Kenny2009.pdfHOME |
Hi,
Thanks for the comments calebu2-ga. However the Chi Squared test you
describe is not what I'm after. I tried a similar Chi Squared myself
by taking the possible vote patterns (1,1; 1,2; 1,3; 2,1; 2,2; 2,3;
3,1; 3,2; 3,3) and comparing their actual number of occurances (1; 1;
2; 1; 2; 3; 0; 2; 5) with the expected number of occurances (17/9;
17/9; 17/9; 17/9; 17/9; 17/9; 17/9; 17/9; 17/9). This gives a p value
of 0.35, and so shows no sign of preference.
However this isn't testing the significance of 6 people preferring the
first
condition, 11 the second, and 17 the third. Is the third condition
significantly more popular than the first? To answer that question I
think I need a test that will take the dependence between the pairs
into account while still testing individuals preferences accross the
conditions. Does that make sense?
I don't think I can do a simulation study. The three conditions are
three versions of an application I've developped and I really don't
know which one users will prefer. Hence why I bought them into the lab
to try three versions out. The application itself is designed for two
friends to use together - hence the paired subjects.
I did also think of proving that the two subjects within a pair are
unrelated? One plan (suggested by a colleague) was to analyse the data
for correlation between the subjects in a each pair, and hopefully
show that there was none. We could then use that as a justification to
assume independence between the pairs. We used a double entry method (
http://users.rcn.com/dakenny/dyad.htm#Top5 ) and Spearman's ranked
correllation rho to get a correlation of 0.22 with a p-value = 0.21.
That's not good reason to suggest a correlation but neither is it good
reason to reject one.
So I think I'm still looking for the right test. It must be out there
somewhere - there's a statistical test for any eventuality ;-)
Thanks,
Tim. A Universal Test of an Expected Utility Theory of War::
by DS Bennett - 2000 - Cited by 33 - Related articlestheir data or theories should apply only to some subset of cases defined .. to use to analyze categorical variables in which there are multiple ically relevant or all dyads.29 When we examine block significance tests, however,
http://www.jstor.org/stable/3014007HOME |
PII: S0962-6298(00)00038-X::
by HPW Toset - 2000 - Cited by 104 - Related articlesto have more armed conflict and militarized disputes than other dyads. We could We also want to test a common assumption in the neo-Malthusian literature, that . also frequently quite marginal and the practical significance of a relationship between .. Categorical data analysis. New York: Wiley.
http://waterportal.sandia.gov/literature/geopoliticsofwater/sharedriver.pdfHOME |
Dumbledad,
I'm not sure if I can sit down and think through the full problem, so
I at least wanted to freely pass on my initial thoughts to whichever
researcher decides to take on the question. (I might come back to it
later, but I normally need a clear mental focus to think through this
stuff :)
I'm also a little bit unsure on a few points, so I figured I'd post
what I had thought up so far and ask some questions to see whether
clarification could prompt further understanding on my part.
Much of what I type comes from knowledge I learned from the following
page of course notes :
Statistics 1B course at Cambridge University (Lecture 9) :
http://www.statslab.cam.ac.uk/~rrw1/stats/S09.pdf
If you are unsure about the independence of the two group responses,
one way to do it would be to arrange the data in the following grid:
1st respondent
1 2 3 TOT
1 1 1 0 2
2nd 2 1 2 2 5
3 2 3 5 10
TOT 4 6 7 17
You then want to test the hypothesis H0: pij = piqj with 0 <= pi, qj
<= 1, sum(pi)=1, sum(qj)=1
vs H1: pij arbitrary s.t. sum(pij) = 1.
The article explains how to do this. The relevant chi squared
statistic is one with 4 d.f.
To answer your main question, I'm stuck on the question of what you
mean by "results are significant"? Do you mean that they are
significantly different from an equal mix ie. pi = p, qj = q?
If so, I believe that what you want to do is test the hypothesis H0:
pij = 1/9 vs H1: pij arbitrary. The d.f. in this case is 8 and the
relevant statistic is the chi-squared distribution.
This approach requires very little assumption about the distribution
of the numbers. If you have some kind of idea of how the two groups
might be related, you are often better off doing a simulation study
and seeing how many times in a group of 10000 trials you get a set of
results that can be categorized by the grid given (or if you decide
what you deem "significant" - a grid of that level of strangeness or
greater).
Let me know whether this is the question you are trying to answer, if
so one of the researchers will give it a closer look (e.g. do the
calculations) - if not, define what you mean by significant and we can
try something else.
Regards
calebu2-ga SHORTEST PATHS THROUGH THE DYADS OF RUMELT'S TAU:::
File Format: PDF/Adobe Acrobat - View as HTMLby DJ Bryce - Cited by 10 - Related articles - All 11 versionsis the categorical method of relatedness identification based on researcher variable supplies a basic test of whether the relatedness component of Only the general relatedness index shows statistical significance in this analysis. The general relatedness index should be a useful tool for assessing
http://www.ces.census.gov/index.php/ces/cespapers?down_key=101771HOME |
Emerald FullText Article : Using patents to benchmark ::
by GS Erickson - Related articles - All 4 versionsPatent counts, of course, have the longest history and broadest use, but patented All other things being equal, patents with more claims should encompass more technology. . To test the significance of these differences in categorical data, . but is far closer to agreement than any of the other diads.
http://www.emeraldinsight.com/Insight/ViewContentServlet?Filename=Published/EmeraldFullTextArticle/Articles/1310030101.htmlHOME |
 Nortel Unveils Vision, Strategy for Israeli High-Performance Net
 Busy Friday Leads to Strong Close for Net Stocks
|
