Trust & Risk project experimental summary Number prefaced with E indicates running order. Number prefaced with P indicates experiment number in the original grant proposal. E1 (P6) advisors equally unbiased E2 (P6) government advisor over-estimates risk E3 (P6) government advisor under-estimates risk E4 (P8) format E5 (P1) accuracy E6 (P4) self-other E7 (P5) status quo E8 (P7) confidence E9 (P3) relative accuracy / self-other E10 (not in proposal) risk perception questionnaire E11 (P9) football paradigm, feedback/judgment format E12 (P10) advisor pics 1 E13 (P2) number of advisors E14 (P11) advice quality 'runs' E15 (P13) basic insurance paradigm E16 phase order (re-run of E4/E18, with explicit phase also at beginning) E17 advisor similarity 3 (re-run of E10) E18 advice format (re-run of E4), with advisors of unequal accuracy E19 Specific information effects on self/other judgments E20 Specific information effects on self/other judgments E21 Advisor similarity 3 E22 Advisor similarity 4 (similarity questionnaire) Experiment 1: Neither advice source biased. Experiments 1, 2, and 3 all utilized the basic advice taking paradigm, which consisted of three phases. In the training phase, participants saw the estimated number of deaths associated with a series of activities, where the estimates were being made by two advisors. One advisor was a (named) governmental advice source, and the other a consumer advice group. Participants noted the two estimates, then made their own estimate, and then were shown the true historical value as feedback. After twelve trials the test phase began, which was the same as the training phase except that no feedback was given. The final phase consisted of a series of questions regarding the bias, accuracy, trustworthiness and values of each of the advisors seen during the course of the experiment. It was possible for us to manipulate the specific levels of bias and accuracy for each advisor (i.e. causing advisors to systematically over- or under- estimate risk, or to have greater or lesser levels of noise built into their estimates). In experiments 1/2/3, the two advisors were equally accurate, but levels of bias between advisors was manipulated. N = 13 for all conditions in all three experiments. Experiment 2: Government advisor over-estimates bias. 2 instruction groups: each n = 13. N = 26. Experiment 2 was the same as experiment 1 in all respects, except that the government advisor over-estimated risk for all of the activities shown, and the consumer advice source consistently under-estimated risk levels. Experiment 3: Government advisor under-estimates bias. 2 instruction groups: each n = 13. N = 26. Experiment 3 was the same as experiment 1 in all respects, except that the government advisor under-estimated risk for all of the activities shown, and the consumer advice source consistently over-estimated risk levels. Experiment 4: Presentation format Overall N = 69 (Sub. 70 removed from group AAD (4) because of incomplete data). AAA (N = 17) AAB (N = 16) AAC (N = 15) AAD (N = 21) This study contrasted probability presentations as % (**a) with those in which the denominator was the same for the two pieces of advice (**b), those in which the numerator was the same for the two pieces of advice (**c), and those in which the denominator was constant across trials (**d). The flaw in this design is that although the denominator same condition changed the denominator on each trial, the numerator condition (group 3) kept the same numerator (1) in each trial, so an additional group was run (group 4), in which the denominator was the same between trials. This minimized the difference between the numerator-same and denominator-same conditions, but made the denominator same condition somewhat artificial and relatively hard to understand. Experiment 5: Advice distinguishable by accuracy. N = 82 BDA (N=40) : gov body more accurate, cons group less accurate BEA (N=42) : gov body less accurate, cons group more accurate All instructions explicit. Mod 4 = 0 or 1 vs mod 4 = 2 or 3 controls the side that gov and con are presented. There are not equal numbers in each mod 4 but there are 20 in mod 4 = 0 or 1 and 20 in mod 4 = 2 or 3. This study contrasted a condition in which the government (sd = 5% of actual risk) was more accurate than the consumer organisation (sd = 20% of actual risk) (*d*) with one in which the government was less accurate (sd = 20% of actual risk) than the consumer organisation (sd = 5% of actual risk) (*e*). Experiment 6: Risk decisions for self vs. other. Based on expt. No. 4 from the original grant proposal, four conditions, probability estimate followed by self/other likelihood assessments (rather than the recommendations to a friend used in E5). Likelihood-of-engagement scores for self and other could be directly compared, and regressions of those scores against risk estimates used to determine whether advice is used differently for self and others. N = 80, 20 per group ADAA - gov more accurate, friend has similar values. ADAB - gov more accurate, friend has dissimilar values AEAA - gov less accurate, friend has similar values AEAB - gov less accurate, friend has dissimilar values Experiment 7: Status quo anchoring. Based on expt. No.5 from the grant proposal, with four conditions: The proposal version of E5 states that Subjects in condition 1 would recommend a friend to start an activity which they have not previously experienced, condition 2 Subjects would recommend that the friend stop an activity in which they're currently involved. There is a confound here - a direct comparison would be C1 start / C2 continue, or C1 avoid / C2 stop. Nigel & Clare liked the idea of incorporating all four conditions - i.e. [1] recommend friend to start an activity not previously experienced, [2] recommend friend to continue an activity already experienced, [3] recommend friend to avoid an activity not previously experienced, [4] recommend friend to stop an activity already experienced. A second (de-conflation) condition was run, in which anchors for the phase 3 slider (as detailed above) were reversed, e.g. continue (max val 15) on left, stop (min val 0) on right, rather than vice versa. N = 112, 28 per group AAAC - advice anchored L>R, recommend friend to start activity. AAAD - L>R, recommend to continue. AAAE - L>R, avoid AAAF - L>R, stop AAEC - R>L, start AAED - R>L, continue AAEE - R>L, avoid AAEF - R>L, stop Note: codes (i.e.code4) were not used to differentiate between instruction conditions in the final program. That differentiation was made by mod4. Experiment 8: confidence in risk judgments Based on Experiment 7 from the grant proposal. Standard design was the same as that used in E4, except that during the testing phase (2) Subjects were shown a range of values from 10% below and above their risk estimate for each activity, and asked the probability (as a percentage) that the true risk falls within that range. These probabilities are effectively confidence ratings (estimates of the probability that their original estimate was correct), and regression can be used to model recommendations for action in terms of confidence ratings in addition to risk estimates. N = 64, 32 per group ADDA - gov more accurate, format = shared denominator, confidence ratings on. AEDA - gov less accurate, other details same as above. Experiment 9: relative accuracy of advisors (and self/other replication). Based on Experiment 3 from the grant proposal. Experimental design was the same as that used in E6, except that the quality of advice from one source was kept static relative to the quality of advice from the other source, between subjects. i.e. Subjects for whom government was a bad advisor may have had good, bad, or even worse advice from the consumer organisation. N = 145 (code2 = D:30, E:28, F:30, G:28, H:29) ADAA - government good, consumer bad AEAA - government bad, consumer good AFAA - both advisors bad AGAA - government bad, consumer even worse AHAA - government even worse, consumer bad Experiment 10: questionnaire study. Previous experiments investigated Ss' risk estimates and judged likelihood of engaging in various risky activities. The likelihood that someone would engage in an activity, however, is almost certainly based on more than estimated risk. A questionnaire was developed to investigate alternative motivational factors in deciding to engage in risky activities. The 25-item questionnaire was presented as part of a Visual Basic program, with the full questionnaire being shown for each of 32 risky activities (8 per each of 4 activity domains). Items were divided into a priori categories, with most items having both a "self" and "other" version. The questionnaire was presented to the same subject population tested by previous experiments. Two conditions were as follows: G1 : code AADC a) Questionnaire without true (criterion) risks of death b) Ss estimate risks of death (entering numerator) c) Ss enter % chance of engagement in activities (if opportunity arose) for both self and others G2 : code AADA a) Questionnaire with 'true' (criterion) risks of death. b) Ss say what their estimates of risk of death would have been c) Ss enter % chance of engagement in activities (if opportunity arose) for both self and others Experiment 11: football paradigm / advice format N = 147 Proposal expt 9. This experiment introduced an alternative paradigm to the project, in which Ss' were asked to judge how many matches (out of three) would be won (or drawn) by the home team in a particular pairing of football teams. Historical data was used. Two advisors were again used (The Sun and Guardian newspapers, known in the UK for divergent political views), and Ss were asked to make Earle & Cvetkovich's trust & value judgments for each advisor. The aim of E9 was to compare the effects of judgments made individually (in series) with those made as a single summary statistic, and the effects of performance feedback given as a series of individual results with that given as a summary statistic. Groups: The Guardian as more accurate advisor: 1. ADDA : single judgment, summary feedback. 2. ADDB : judgments and feedback both in sequence. 3. ADDC : single judgment, sequential feedback. 4. ADDD : sequential judgment, summary feedback. The Sun as more accurate advisor: 2. AEDA : single judgment, summary feedback. 3. AEDB : judgments and feedback both in sequence. 4. AEDC : single judgment, sequential feedback. 5. AEDD : sequential judgment, summary feedback. Participants each made 15 training judgments, and 15 test phase judgments. Experiment 11 employed the same method for perturbing advice (i.e. drawing samples from distributions with unequal variance) as did earlier TrustRisk experiments, but the final advice numerator must be a 0,1,2, or 3 (i.e. number of matches won or drawn by the home team, out of 3, rather than an estimate of risk). Advice from the two sources were converted to that format in the following way: If an estimate deviated from the true (criterion) value by less than 10%, the true value was displayed. The next two steps of deviation in advice quality were at 30% and 70%. For each step, one point (i.e. one match) was added or subtracted (in the same direction as the original deviation), to a maximum of 3 or a minimum of zero. Therefore, if X(1) = (criterion + 11%) then X(1) is increased by 1. (crit - 31%) would decrease by 2. 112 of the 147 participants were female, leaving 35 male participants. The age range was 17-32, median age 18. Experiment 12: advisor similarity E10 employed the standard risk behaviour paradigm. Advisor identity was manipulated along two dimensions: broad physical appearance (sex, age) and stated values. Ss were asked to respond to 5 simple values statements, which the advisors' own value statements were then based upon. Advisors were grouped into comparatively young (up to 30, approximately)/old (middle age +) categories. Ss presented with a physically similar advisor would see one of the same sex and general age category as themselves (30 being the cutoff), and similar-values advisors had statements shown next to their photographs which exactly matched the Ss' own stated values. Advisors who were different on either dimension were diametrically opposed to the age/sex category or value-responses of the subject. Experiments 12, 14, and 19 had no self/other phase - just 16 trials per training, testing phase, and an explicit judgment phase. Conditions: AADA : SLSV(similar looks, similar values)-government, DLDV-consumer AADB : SLSV-consumer, DLDV-government AADC : DLSV-government, SLDV-consumer AADD : DLSV-consumer, SLDV-government i.e. ignoring the government/consumer factor for a moment, there are effectively two groups here.. [similar values & appearance vs. different] and [contradictory components]. Although simply looking at groups AADA/B should show any overall effect of similarity, analysis of all four groups allows us to tease apart the influences of the advisors' physical appearances and stated values on trust, as shown below: sim looks dis looks sim vals AADA gov AADC gov AADB con AADD con dis vals AADC con AADA con AADD gov AADB gov Experiment 13: number of advisors Based on Experiment 2 from the grant proposal. E13 employed the standard risk behaviour paradigm. Participants saw either zero, one, or two advisors. All participants saw advice in percentage format, advisors had equal accuracy. Codes were AADA (zero advisors), AADB (one advisor), and AADC (two advisors). Experiment 14: accuracy switching (good to poor advice, or vice versa) N = 112 (4 x 28) This experiment differs from the standard trust-risk advice taking paradigm in that advisors' accuracy is not necessarily constant: the two advisors may swap accuracy values midway through a 32-trial training/test phase. Code4 (which controls direction of advisor accuracy switching) modulates code2 function. If code4 = B or C, then accuracy value of advisors switches at trial 17 (of 32). Group codes: ADDA - gov advisor more accurate, no switching AEDA - gov advisor less accurate, no switching ADDB - gov advisor more accurate, then gov advisor less accurate AEDB - gov advisor less accurate, then gov advisor more accurate Experiment 15: insurance task paradigm The intent of Experiment 15 was to introduce a variant of the advice taking paradigm to the project. In this paradigm there are two named advice sources, and 15 trials in both the training and testing phases. There is no self/other or recommendation phase. On each item, in addition to making risk judgments participants made an insurance purchase decision, and outcome feedback regarding that decision affected participants' monetary resources (which were used to buy insurance or pay for housing costs). Participants chose the amount of insurance they wished to buy, based on the predictions made by two advisors (one similar, one dissimilar). There were 20 trials and three between-subjects conditions, large difference in advice quality between good advisor and bad advisor, small difference, no difference. Experiment 16: experimental phase order N = 60 (2 x 30) Experiment 16 was exactly the same as experiment 6 (basic paradigm), except that there were explicit judgment phases at both the beginning and end of the experiment. This allows comparison with the results of other experiments, to check the apparent influence of advisor accuracy etc on previous explicit judgment measures. ADAA - gov more accurate. AEAA - gov less accurate. All participants saw two explicit judgment phases. Experiment 17: advisor similarity 2 The lack of significant results from experiment 12, and certain comments from participants, led us to believe that Ss may not have been paying any attention to the similarity information given in association with each advisor. Subject groups were the same as those in E12. A modified version of the experimental software was developed which included two boxes per advisor, which Ss were required to tick/untick before moving on to the next item (hopefully forcing them to acknowledge the advisor's similarity characteristics). The first box asked "sex and age similar?", the second "values similar?" Experiments 12 and 17 had no self/other phase - just 16 trials per training, testing phase, and an explicit judgment phase. The second code letter refers to accuracy by advisor type (D = government advisor more accurate, E = consumer advisor more accurate). Groups run in E17: (D = dissimilar, S = similar, L = looks, V = values). ADDC : DLSV-government, SLDV-consumer ADDD : DLSV-consumer, SLDV-government AEDC : DLSV-government, SLDV-consumer AEDD : DLSV-consumer, SLDV-government 98 people participated in this experiment. Ages ranged from 17 to 43 years (median = 19 years). 81 participants were female. 18 were male. Participants always saw unbiased sources. However these varied in accuracy. For some participants the government gave better advice than the consumer organization, for others the consumer organization gave better advice. Experiment 18: Advice format Expt 18 is a variant of E4 (Advice format), in which the advisors are of unequal accuracy and 4 risk domains are used rather than 3. Only the original three format conditions are to be run (percentage, numerator matched, denominator matched within trials), which combined with the two accuracy conditions gives a 3x2 design. 30 participants per group (total n = 180). In E4, the 4th group run (denominators matched both within and across trials) was an attempt to resolve a confound between conditions 2 & 3 (i.e. confound between either numerator or denominator being fixed, and between the fixed element changing between trials or not). Dr Harries now feels that this confound is not a problem, as the two components run in different directions, one facilitating an effect and another inhibiting it. Code4 always set as "C" (i.e. 2 advisor setting from E13). Gov advisor more accurate: ADAC - percentage format ADBC - denominator matched ADCC - numerator matched Gov advisor less accurate: AEAC - percentage format AEBC - denominator matched AECC - numerator matched 33 people participated in this experiment. Ages ranged from 18 to 25 years (median = 19 years). 29 participants were female. 4 were male. Participants always saw unbiased sources. However these varied in accuracy. For some participants the government gave better advice than the consumer organization, for others the consumer organization gave better advice. Experiment 19: Specific information effects on self/other judgments N = 32 Previous research (Weinstein, 1982) has indicated that individuals exhibit a tendency to make different risk responses for themselves as opposed to someone else. The current study examined if additional information about a second person could influence individuals' estimations of that person's risk preference. The second person was seen as less likely to engage than self in both types of activity when presented as a low- risk taker, a result that was reversed when the other was presented as a high- risk taker. Additionally previous experience of a risk was found to mediate likelihood of engagement. Experiment 20: Specific information effects on self/other judgments When taking advice in the real world, people are often exposed not only to directly experienced information regarding the accuracy of advisors, but also third-party information regarding that accuracy. Experiment 20 had a 2X2 design, with two between-subjects factors: (1) actual accuracy of the two advisors (government better or worse than the consumer advisor), and (2) reported accuracy of the two advisors (again, gov better or worse). Experiment 20 codes were ADDC, AEDC, BDDC, & BEDC, where the second letter codes for advisor accuracy (D = gov better, E = gov worse), and the first letter codes for explicit information regarding advisor accuracy (A = info says gov better, B = info says gov worse). Experiment 21: Advisor similarity 3 Experiment 21 was a replication of Experiment 17 (advisor similarity), except that a photo was provided when participants were asked to judge their explicit attitudes toward the general categories of government or consumer advisors (in Experiment 17 no such photo was provided except when participants were judging for specific gov't or consumer advisors). The 'general' category advisor photo was drawn randomly from among the appropriate-source (government or consumer) advisors which had been seen during the experiment. Experiment 21 also included an explicit measure of whether participants would use the information provided by advisors. Data collection and analysis for experiment 21 continues. Experiment 22: Advisor similarity 4 (similarity questionnaire) In addition to a limited replication of experiment 17 (advisor similarity), experiment 22 introduced a brief questionnaire (below) in which participants were asked about the similarity of the advisors seen during the experiment. None of the questionnaire answers correlated significantly with stated trust in the advisors. "Please use percentages from 0 - 100% to rate how much you agree with the following statements, where 0 % is complete disagreement and 100% is complete agreement." 1. The advisor is of a similar age to me. 2. The advisor is of a similar level of attractiveness as me. 3. I believe that the advisor has similar educational qualifications to me. 4. I believe the advisor has a similar IQ to me. 5. I consider the advisor and myself to have similar backgrounds. 6. I would befriend this advisor. 7. I would expect this advisor to have similar hobbies to myself. 8. The advisor has similar values to me. 9. I would trust this advisor to be competent in handling my affairs. 10. I believe this advisor would have my best interests at heart. 11. I would ask this advisor for advice concerning personal matters. 12. The gender of the advisor is the same as the person I take most of my advice from. 13. The advisor is physically similar to someone I would usually take advice from. 14. The values of the advisor are similar to someone I would usually take advice from.