Psy 2113 Unit 2 Notes.docx

PSY 2113 Unit 2 Notes MEASUREMENT Design Terms  Units o The objects on which the experiment is done  Response Variable (Dependent Variable) o A variable whose changes we wish to study: an outcome or a result  Explanatory Variable (Independent Variable) o A variable that influences or explains a response variable  Treatment o Any specific experimental manipulation applied to units  Lurking Variable o A variable that has an important effect on the relationship among the variables in the study, but is not included among the variables studied  As ice cream sales go up, drowning rate also increases  Lurking variable = temperature increase  Confound o Two variables are confounded when their effects on a response variable cannot be separated easily from each other  Variables difficult to remove Issues with Measurement  When asked about income, people may… o Intentionally lie about income o Not know their actual income  Not remember details (Tucson Garbage Study) o Self-reports of alcohol use and estimates by rummaging through garbage cans were way off  May not understand questions and answer anyway o Jimmy Kimmel – Lie Witness Investigation  May involve sensitive issues o Drug use and sexual activity Measurement  We measure a property or a characteristic of a person when we assign a number to meaningfully represent that property of the person o Examples:  Height and weight  Depression  Intelligence  Sex

The Many Ways of Measuring  Incidence Rate o Number of new disease cases per year  “In 2016, there were X cases of chicken pox  Prevalence Rate o Incidence rate per 100,000 people  “In 2016, there were X out of 100,000 people who had chicken pox Lingering Confusion  Consequence of measuring  We can truthfully say: o The incidence rate of new HIV infection is rising o The prevalence rate of HIV infections is dropping  How can this be? o There are more and more new cases every year, but…  The population of non-infected individuals is growing faster than the incidence of the disease  This is a critical reason why you need to take a closer look at the data being used to back up popular claims Measuring Academic Preparation for College  High school grades o What high school? o Do all teachers grade the same? o Do all classes cover exactly the same material in exactly the same depth and breadth  Standardized Test Scores o Opportunity to Learn (OTL)  Does everyone have the same chance at test preparation? o Do some individuals test less well under pressure? o Are certain tests culturally biased?  Advanced Placement Courses o Does every HS student have access to these courses – or only the more wealthy districts?  AP exams o Teaching to pass the test or to learn the material? o How does the test translate into college material?  Work portfolio o Representative? o Accomplished with the assistance of others? o Time-consuming to judge  Letters of Recommendation



o Argument from Authority Extracurricular activities o Concern is the curricular activities o Quality of contribution

Conclusion on Measuring Academic Preparation for College  No one perfect measure  Use many such “imperfect” measures for bias-cancelling properties What does this show?  Absolute vs. Relative Judgement o Best available choice  Vastly different answers o 83% of students unhappy with parking at OU o 91% chose option 4  Force a similar context for comparison  Establish a common “metric” Valence:  What percentage of Americans think the US government spends too much money on assistance to the poor?  What percentage of Americans think the US government spends too much money on welfare?  Valence, as used in psychology, especially in discussing emotions, means the intrinsic attractiveness (positive valence) or aversiveness (negative valence) of an event, object, or situation  Happy words have positive valence  Threatening words have negative valence  Ideally, we should use emotionally neutral words when asking survey questions Closed vs. Open-Ended Question Formats  Vastly different answers  Recall vs. Recognition o Recall = absolute measurement o Recognition = relative measurement  Usually easier Cognitive Priming and Order  1 = very unhappy to 5 = very happy o How happy are you with your GPA? o How happy are you with your life?  How happy are you with your life?  How happy are you with your GPA?



Each question may prime a different response which will affect the answer after it, sometimes can cause an Order Effect

Priming Principle  Gestalt or Holistic Assessment: o Whole is not the sum of the parts  You can be unhappy with your GPA, but you can still be happy with your life  Reductionistic Analysis: o Whole is sum of the parts  If you are unhappy with your GPA, then you are also unhappy with your life  People are primed by whether or not you ask them first for a holistic assessment or a reductionistic assessment Scale Anchoring  Indicate the degree of satisfaction with instruction o Scale 1: -2 -1 0 1 2 o Scale 2: 0 1 2 3 4  What will happen? Implicit Norming  In a typical week, on how many occasions do you have at least 2 drinks in one sitting? o Scale 1: 0 1 2 3 4 o Scale 2: 0 1 2-3 4-6 7+  What will happen? o Scale 2 makes it appear that 4-6 drinks are nothing unusual  Only 7+ is the extreme category o Participants assume that survey knows the NORMS surrounding a typical amount of drinking and designs the scale accordingly Special Considerations  Must establish common metric  College Rankings (US News) [Issues for this measurement] o Acceptance rates  Number accepted / Number applied  Solicit more applicants without much chance of admission  Count waitlisted students as accepted?  Transfers? o Yield rates  Number accepting / Number offered  Early admission procedures  Guarantee of admission conditional upon guarantee of acceptance

o

o

o

o

 Hurts those with less money to apply  Hurts those with need of financial aid Average SAT scores  Average of all acceptances  SAT taken multiple times – which score to count?  Special admissions? Sometimes not counted  Most universities reserve 8-10% of admissions for special circumstances in which ACT does not apply  International students?  Great math scores, poor verbal scores Graduation rates  Percent graduating in 5 years  Transfer in or out?  When does the time-clock start and stop?  Hours required for graduation  Duke only requires 96 hours for graduation, as do many private schools Reputation  Other-report ratings scale  Little change in reputation despite clear changes in schools Student/faculty ratios  Number of students / Number of permanent faculty  Case Study 1 o Duke University  4,500 students / 1,200 faculty  Ratio = 3.75  Case Study 2 o University of Oklahoma  20,000 students / 1,500 faculty  Ratio = 13.33  How many faculty actually teach?  Duke = Average course load = 1.5 / year  OU = Average course load = 4 / year

Changing Measures Over Time  Report on sexual assaults  National Crime Victims Study (NCV)  After 1991, added the following question: o Have you ever had sexual intercourse when you didn’t want to because of alcohol or drug?

RELIABILITY & VALIDITY Measurement Principles  Two key concepts: o Reliability  Consistency of measurements o Validity  Conceptual and practical meaning

Reliability:  A measurement process is said to be reliable if: o Repeated measure on the same object or individual give approximately the same results o Ideas is one of consistency o High precision in measurement means little variability in measurement  Same result every time How to Obtain Reliable Measurement  Some things to be measured don’t vary much o Height, reaction times, reading speed  If things to be measured are inherently variable… o Average over many measurement  Aggregated (AVERAGED) measurements are almost always better than single measurements o Examples:  Blood pressure  Not once, but on multiple occasions throughout the day  BP varies by time of day, depending on age of patient  BP varies by psychological discomfort (Fear of White Coats)  Classroom examinations  1 questions per exam?  10 questions per exam?  25 questions per exam? o Multiple measurements averaged together to get an overall score Reliability Measures?  Reliability ranges between 0 and 1 o 0 means no consistency (completely unreliable) o 1 means perfect consistency (completely reliable)  Within psychology, we tend to be happy if our reliability is above 0.7

Typical Reliability of Common Psychological Measures  Personality = 0.8  Intelligence = 0.97 o MOST RELIABLE  SAT = 0.93  Structured clinical interview = 0.65  MMPI Diagnostic test = 0.73  Unstructured clinical assessment = 0.55 o LEAST RELIABLE Medical Diagnosis  Medicine is often referred to as both an art and a science  Due to its complexity, medical diagnosis has a large degree of variance within it  Many medical diagnoses are less reliable than psychological diagnoses o Example:  1978 study of 13 radiologists revealed an average reliability of only 0.15 among all 13 pairs of radiologists when reading the same image o Technology can help us overcome this, but not entirely o This is why most doctors use consults and second opinions Validity Definition  A measurement of some property of an object is valid to the extent that is relevant or appropriate as a meaningful representation of that property o Ex: A ruler measures height o  Is shoe size a valid measure of IQ? o No Validity  Reliability is a prerequisite for validity o Just because a measurement is reliable, does not mean that it is valid  Ex: scale that is always incorrect by 5 pounds  Reliable = yes  Valid = no  If you can’t get consistent measurement, then you cannot get valid measurement  Reliability is necessary, but not sufficient for valid measurement Three Types of Measurement  Content Validity: o Does the measure contain an appropriate sample from the domain of intended content?  Covers all the information you want it to cover





Ex: Final Exam o Covers unit 1, unit 2, unit 3, unit 4, and unit 5 o Typically assessed by asking Subject Matter Experts (SMEs)  An authority on the subject o Examples:  Statistics course  What items should appear? o Material from every powerpoint  Depression scale?  What questions should appear? o Sleep problems, moodiness, sadness o Suicidal thoughts, alcohol use (male), crying (female)  Who decides? o Subject Matter Experts (SMEs)  Measure of Academic Preparation for College?  What should be measured?  Do you agree with this list? o Reading comprehension o Reading speed o Extensive vocabulary o Basic knowledge of geometry o Basic knowledge of algebra o Problem-solving skills Predictive validity: o Can the measurement be used to predict future performance? o Usually assessed by a correlation:  -1 indicates inverse predictive validity  Perfect negative relationship  As days of missed class go up, quiz scores go down  0 indicates no predictive validity  No relationship  Hair color and intelligence  1 indicates perfect predictive validity  perfect positive relationship o As ACT goes up, so do graduation rates o Examples:  Use SAT to predict college success  Use personality test to predict job performance  Use HS GPA to predict college performance  Predicting Graduation Rates  42% of all students with ACT score of 21 graduate within 5 years  84% of all students with ACT score of 24 graduate within 5 years  Should we use ACT scores to select for admission?



o Yes… if our goal is to increase graduation rates  ACT scores are clearly related as twice as many people graduate with ACT = 24 than with ACT = 21 o No… if graduation is not a criterion of interest or a priority for the college  Some ACT = 24 scorers will not graduate (16%)  Some ACT = 21 scorers will graduate (42%) o No prediction is ever perfect, but some are useful o Mistakes of prediction will happen and are the price of using any selection system  Sometimes you will swipe right, when you should have swiped left, and vice versa Construct Validity o Does the weight of the evidence support interpreting the measure as indicative of what we mean by the construct?  Does the test for Depression actually test for Depression?  Or is it measuring something else, like loneliness?

Potential Validity Checks  Does IQ correlate with: o Achievement o Educational attainment o Processing speed o Other tests of intelligence o Life success o Level of brain activity o Adaptation to novel environments o Analytic problem solving o o o o o o o o

Language of origin Gender Ethnicity Temperament Culture Age Motivation Activity preferences

Conclusion on Validity  It’s a matter of degree  No measure is absolutely valid or invalid  Some measures are more or less valid than other measures of the same construct  Try to use the most valid measure

Computing Rates & Percent Change Computing Rates  Rate: count per unit of population  Rate = Proportion X per unit of population  Suppose there are 62 women in a class of 82 o Proportion = 62/82 = 0.756 o Percent = 62/82 x 100% = 75.6% o Rate per 100 = 62/82 x 100 = 75.6 per 100 Usefulness of Rates  949 bicycle deaths  24.8 million riders o Proportion = 949/24.8M = 0.0000383 o Percentage = 949/23.8M x 100% = 0.00383% o Rate per M = 949/24.8M x 1M = 38.3 per million o Rate per 100K = 949/24.8M x 100K = 3.83 per 100K  Proportion and percentage can be too small to for people to grasp o But not other rates  Rate per 100K o Think of the number of people at Memorial Stadium Percent Change  A stock market index fund resulted in a 30% gain. Assuming $100k investment, how much money do you now have? o % Change = (amount of change / starting value) x 100  30% = (x/100,000) x 100  x = (30 x 100,000) / 100  x = $30,000 net gain  Measurement is important and difficult  Science is measurement  Beware of fuzzy math with rates and percentage increases

CPI and Inflation Consumer Price Index  Index number that allows us to adjust for known effect of inflation  CPI = (Value / Base Value) x 100



Current CPI uses the years 1982-1984 as the base value that is set to 100 to make the math easier

Main Idea  Dollars are not constant – they actually lose or gain value over time  Need to make an adjustment for this “changing metric” over time  CPI is an overall “Price Index” o Compare prices of items over different years Percent Increase/Decrease  Compare the new CPI to the baseline of 100 o If your new CPI is 100, then there is no change o If your new CPI is 150, then the price of the goods is 50% higher than in 19821984  (150-100 = 50)  (New CPI value – Old CPI value) = percentage difference o If your new CPI is 50, then the price of the goods is 50% lower than in 1982-1984  (50-100 = -50)  Example: o The cost of tuition, room, and board (TRB) at a 4-year public institution of higher education in the US was $4,708 in 1982. o In 2018, it cost $22,126 for TRB at OU o How much does the TRB cost in 2018, compared to the relative cost in 1982?  Making the CPI Adjustment o The tuition, room, and board price index (TRB-PI) number in 2018 using the year 1982 as a baseline is 22,126 / 4,708 x 100 = 4.70  Interpretation: the costs of attending college has risen by 370% over the 36 years since 1982 Value of CPI  Question: Is the cost of TRB faster or slower than the overall rate of inflation? o Faster  Current CPI is 241.12 (December, 2017)  CPI measures a consumer’s purchasing power from year to year and allows one to make meaningful, over-time comparisons Fixed Market Basket Price Index:  An index number for the total cost of a fixed collection of goods and services at a given point in time  Provides an anchor or common frame of measurement on which to calculate the change in inflation o Step 1: Sampling of Typical Purchasing Habits



In 1982, the BLS measured the typical spending habits of 29,000 households sampled with a probability-sampling model  They measured these spending habits on over 400 items, and these are used to gauge the amount spent by the average household in a given year o Step 2: Constructing a fixed market basket  Spending habits are roughly categorized into 8 areas for specific comparisons  8 Major Purchasing Areas: o Food and Beverages o Housing o Apparel and upkeep (includes jewelry) o Transportation (includes traffic fines) o Medical care o Recreation and entertainment o Education and communication (tuition, telephones, computers) o Other miscellaneous (haircuts, bank fees, etc.) o Step 3: Every year, a new total price for those goods is determined by sampling prices in retail outlets, grocery stores, etc.  The total cost of these goods in a future year is compared to the cost of the goods in 1982, and this cost is used to calculate the CPI  Recent Examples  2012 = 229.6  2013 = 233.0  2014 = 236.7  2015 = 237.0  2016 = 240.0  2017 = 241.1 o This means that the cost of goods in 2016 is 140% higher than in 1982-1984. o This applies to the same basket of goods and services in 2016 as in 1982 o Another way of saying this is that, it would be $2.40 to buy the same thing in 2016 that it took $1.00 to buy in 1982 CPI Changes from 2002-2013  Since 2002, constant, steady rise; and small plateau until drop 2009  Large drop in 2009  Constant, steady rise since 2009 Who Decides What Goods and Services to Include in the CPI?  Controversial for measuring inflation

 

   

Headline CPI: o The standard CPI reported by the media, which includes the 8 categories given previously Core CPI: o Typically removes fuel and food from the CPI basket, because they are too susceptible to supply and demand shocks, and are thus too variable to guide policy Core (Red) vs. Headline (Blue) o By including gas and food, the CPI is less stable (blue) o By taking them out, the CPI is more stable (red) Gas Prices – Unadjusted o Less stable Gas Prices – Inflation Adjusted o More stable o Gas has become about the same price across time Compared to Core CPI o The CPI has increased a lot o But gas prices have been steady  -> Gas should be cheaper to purchase now, than it was in 1982

A Valuable Re-expression:  To re-express a price in current dollars and adjust for changes in buying power, we can use the following conversion formula: o $ at Time B = $ at Time A x (CPI at Time B / CPI at Time A) Physician Incomes  In 1981, the median physician income was $89,900. In 2018, it is $187,200.  The CPI in 1981 was 90.9; it is 241.1 today. In terms of constant dollars, the physician income should have been: o Y = 89,900 x (241.1 / 90.9) = $238,448 in the year 2018 o 187,200 – 238,448 = -51,248 Net Gain / Loss  The net loss in current dollar compensation over the 37-year period after adjusting for inflation is $51,248 Cautions:  CPI does NOT measure the cost of living o One can always live cheaper by doing with less  Cannot reflect personal spending habits o You don’t have to buy what everyone else does  Must somehow account for changes in quality o Computers today are better than computers yesterday



Must account for new products o iPads in 1981?

Quality and New Innovations:  Space flight  DNA  Vaccine for Polio  Color Television  Cable TV  Cell phones  Personal Computers  Internet Cost of Living:  One can always influence their cost of living by doing one of two things: o Require more things – or more expensive things – for living o Require fewer things – or cheaper things – for living  Example: o 1895 Montgomery Ward catalog: o At the turn of the century, Montgomery Ward was the largest mail-order business in the US o It was one of the only ways that the forty percent or so of America’s households could purchase the products of industrial civilization  Time Needed for an Average Worker to Earn the Purchase Price of Various Commodities:

o o Bottom Line: Fixed Market Basket







If you average over all the commodities they made then and what we make now, you find that the average productivity multiplication is about tenfold An average worker today could buy with one hour’s work the average bundle of things that an average worker of a century ago took 10 hours to earn However,  This set of calculations (taking commodities that existed then and exist now and comparing their labor-standard prices) is conceptually flawed  We make many things today that were not made back in 1918. And that broader range is not factored into the calculations anywhere

For Your Consideration  In Looking Backward, Edward Bellamy’s utopian novel, the narrator – thrown forward in time from 1885 to 2000 – hears the question, “Would you like to hear some music?”  He expects his host to play the piano – a high social accomplishment around 1900. To listen to music on demand then, you had to have an instrument and someone to play it.  It would have cost the average worker some 2,400 hours (roughly a year) at a 50-hour workweek to earn the money to buy a high-quality piano, and then there would be the expense and the time committed to piano lessons  Today, to listen to music on demand in your home, what do you need? o The internet, a cellphone, Spotify, etc. (much easier now, than before)  Of course, the experiences of live and recorded music are different in kind. Are they different enough to put a serious dent in the fact that a household today can acquire the capability of listening to piano music for only 1/240 the labor time that it cost a household a century ago? Current Thinking on CPI  In a rapidly-accelerating high-tech society, the idea of a fixed market basket of goods and services loses meaning every decade or so  That means a re-settling of the fixed market basket would have to take place every ten years or so Paper 1:

DESIGN & ETHICS Validity

Design and Validity  External validity: o Do results generalize beyond the sample to the population of interest? o Ensured by good sampling  Internal validity: o Is the data sufficient for drawing cause and effect conclusions?  Construct validity: o Is the measure a true measure of the construct of interest? Random Assignment vs. Random Sampling  Random Assignment (also called randomization) to groups is the most important part of experimental design (how you put people into groups) o It is the only strategy which controls for lurking variables o Controls for lurking variables by allowing us to assume that all variables are equivalent, save the treatment differences  This is a different concept from random sampling  Random Assignment increases internal validity of a study by controlling for lurking variables, thus allowing us to assume causality  Random Sampling increases external validity by achieving a representative sample which adequately reflects the population Random Assignment: How Many Groups?  At least two o But may have more, if the design requires it, and resources allow  Standard clinical trials have three treatment arms: o Active drug group (A): Receive the treatment o Placebo group (P) (control): Those who receive a placebo which simulates the drug, but doesn’t actually do anything o Natural history group (NH) (control): Have no treatment of any kind, and whose condition is allowed to run its natural course  Why? o Three comparisons are made  Efficacy of drug: Difference between A and NH, which shows the efficacy of the drug’s treatment  Efficacy of active drug’s active ingredient: Compare A to P  Magnitude of the placebo response: Compare P to NH o Types of effects:  Specific effects of active ingredients  Non-specific effects: Drug side effects  Unintended effects: the pure placebo effects which may indicate a complex psychobiological phenomenon, contingent upon the psychosocial context of the subject, that may be due to a wide range of neurobiological mechanisms









Generic bio-psycho-social health-related effects: like weather, occupation changes, diet/lifestyle habits, emotional changes, etc.  Serendipitous effects: the effects of being in therapy, such as the ritualistic (organizing) effects of the therapeutic structure, inadvertent role-taking, chance remarks or encounters, the influence of auxiliary personnel, and motivational components  Natural history effects: different than placebo effect. Shows the natural termination of the disease. Helps us know if there may be temporary improvement that could be confused with a cure. Shows the cycle of the disease o Placebos control for non-specific side effects, allow us to monitor placebo effect, and for serendipitous effect (i.e. just being treated helps) o Natural history of the disease and general biopsychosocial effects are controlled for by randomization Other Experimental Issues o Blindness to treatment condition  Clever Hans the Famous Horse o Single vs. Double-Blindness  Should doctors be blinded to treatment conditions? o Refusal to participate  Electro Convulsive Therapy (ECT)  Because of its bad reputation, only high SES people participate in studies o Non-adherence to experimental protocol  Alternative medicine  Interferes with many active ingredients in prescription medicines o Placebo-group contamination  Diffusion of treatment to those in the placebo group o Fidelity of treatment conditions  Correct application of clinical protocol An Example: o Study 1:  Randomly sample from a target population of older men  Ask these older men – retrospectively – if they take aspirin  Compare heart attack rates between those who have had taken a lot of aspirin vs. those who have not  Findings:  Those who take more aspirin have fewer heart attacks  Conclusion:  Taking aspirin prevents heart attacks o Any problems with this study? Issues with Measurement o Measurements of key outcomes is a bit tenuous  Recall is very hard to do





o Do those who take aspirin on a regular basis differ from those who don’t in some important way? Group Differences o Better fitness  Take aspirin to reduce inflammation from exercise o More health conscious  More aware of their bodies’ general health o These differences, if they exist, may make the conclusion that an aspirin a day prevents heart attacks unwarranted o It is possible that those who are in better shape and are more health-conscious also tend to have fewer heart attacks Another Example: o Study 2:  Physician’s Aspirin Study  Half of 22,000 male doctors were assigned (at random) to take an aspirin every other day  The other half were assigned a placebo  After 5 years, 239 in placebo group had suffered a heart attack; only 139 of these aspirin group had  Study Schematic



 Conclusion:  Taking an aspirin every other day reduces the risk of having a heart attack  taking half and aspirin a day seems to be preventative for heart attacks  Random assignment to groups assures us of a high likelihood that the groups were similar BEFORE the introduction of the treatment (Aspirin)

TYPES OF DESIGNS Observational Study  Observational Study o Observes individuals o Measures variables of interest o No active attempts to influence responses



Medical example: Epidemiological study

Experimental Study  Experimental Study o Deliberately imposes treatment on individuals o Measures response variables of interest o Looks for differences between Treatment and Control groups  Classical example: Scientific study  Medical example: Clinical trial or Drug trial Randomized Comparative Experiment  4 key components o 1. Use random assignment to conditions o 2. Control effects of lurking variables through randomization to groups and by comparing several treatments (e.g. using a control group) o 3. Replicate experiment by using many subjects per condition o 4. Assess statistical significance – Rule out coincidence as an explanation Example  Is poor attendance a cause of poor grades?  They are associated o Correlation typically ranges between .50 and .90  Attendance -> Grades Question the Causation  Potential lurking variables? o Achievement Motivation (AM)  Achievement Motivation -> Attendance  Achievement Motivation -> Grades How to Assess Lurking Variables  Divide groups into High AM and Low AM  What is the effect of class attendance within each group?  If it is zero (within the margin of error), then the relationship is zero when statistically controlled  We may then conclude there is no causal relationship between grades and attendance when controlling for achievement motivation



Problems?  For those in High AM, there may be no participants with low attendance o No way to compare within the group  For those with Low AM, there may be no participants with high attendance o No way to compare within the group  This is what we mean by confound (when you can’t separate/compare people into groups)

 Experimental Idea  Randomly assign one group of students to show up for class  Tell the other half they cannot show up  Observe responses and compare treatment to control  Problems? o Key Ideas of Design  In principle, experiments can give stronger evidence for causation  Experimental Study – High internal validity o Directly applying the differences between groups  We can be more confident in cause-and-effect conclusions  Observational study – Low internal validity  Observational studies are much weaker than experimental studies  Correlation is necessary, but not sufficient condition for causation

DESIGN IN REAL LIFE Terms of Usage  Efficacy o Effects under ideal, controlled conditions (clinical trials)  Effectiveness: o Real-world results o Abecedorian Early Intervention Project o Active-Listening marital therapy Most Famous Example  New Coke

o All randomized, double-blind experiments showed people preferred New Coke to Classic Coke o Real world results? RESEARCH ETHICS  Research Ethics o Belmont Report (1999) o APA Ethics Code (2003) o Institutional Review Boards  Universities  Those receiving Federal Research Money Four Basic Principles:  Informed Consent: o Subjects must consent – and be able to consent – to participation  Confidentiality o Results will not be shared with any unauthorized individuals  Interest of Participants o Interests of the participants must prevail over any and all other interests  Cost/Benefit Analysis o Is the study worth it? Research on SBS  Magnetic Resonance imaging (MRI) as a diagnostic tool for Shaken Baby Syndrome (SBS) o Parents show up at ER with injured infant o Infant shows signs of SBS o More definitive diagnosis might be obtained with MRI o Sedation required for MRI (risky to an infant with neurological complications) o Only parents can give informed consent  Does this pass or fail the ethics test? o The babies cannot give consent o Not putting the participant first o Are parents the best to give consent for the baby?  We don’t know how the baby may have gotten SBS o Sedation is risky (cost vs. benefit) Stem-Cells and Parkinson’s  Nature of placebo o What would be an effective group for surgical implantation?  Issues of blindness o Wouldn’t everyone know who got the surgery?  Should sham surgery be performed? o Do the surgery, but without the stem cell implantation



What would happen otherwise?

Stanley Milgram – A Study of Obedience to Authority  One subject and one confederate  Confederate hooked up to a shock device  Subject told to help “train” confederate by administering shocks upon experimenter demand  Shocks got more and more sever  But shocks were simulated  What happened? o Subjects in the study gave apparently “lethal” doses o Some were scarred forever by that knowledge o Does this pass the ethical test?  NO Sherif and Sherif  Studied intergroup relations, prejudice, and stereotyping o At the University of Oklahoma  They were interested in the dating and sexual practices among youth of different ethnic backgrounds  Hung out at a drive-in movie theater  Administered surveys and collected “observational data”  Does this pass the ethics tests? o NO  Informed consent not administered Social Psychology  Pretending to be a member of a group to study behavioral patterns  Rosehan (1960) – “On Being Sane in Insane Places” o Pseudo-patients  Fake admission to hospital to gather data on mental hospital practices  What do you think happened?  Pseudo-patients got diagnosed with schizophrenia  Is this ethical to the participants? To the institutions?  NO o Don’t have the ability to give informed consent o Doctors have trust issues with patients vs. pseudo-patients Stored Blood and Genetic Samples  Can researchers use human samples collected years ago for another purpose and without informed consent? o NO.



What if the person is dead, as well as all of their ancestors? o NO?

AIDS Trials in Africa  Clinical trials on patient populations that cannot afford the eventual treatment  Vaccines developed on patient populations that would not eventually get the vaccine  Is this ethical? o NO Surveys of Youth  Ask about pubertal development  Ask about sexuality and onset of first intercourse  Should consent of parents be required? Deception of Subjects  Lotane and Darley – Good Samaritan Study o Used seminary students to assess situational factors on helping behaviors o Had a confederate fake being “drunk” on steps o What did these students do under different circumstances?  Differences between religions?  Difference between time constraints? Payment for Research Participation  Pay a person in poverty to be a research subject  Is this inherently coercive?  What about: o Prisoners o Children o Students o Workers

DESIGN ISSUES IN DRUG DEVELOPMENT FDA Process  Continually evolving  Meticulous examination of design protocols  Results in few drugs being approved Purpose  All new drugs need proof that they are effective, as well as safe, before they can be approved for public use  No drug is absolutely safe



o There is always some risk – hopefully small – of an adverse reaction Cost/Benefit Analysis o The FDA determines that when the benefits outweigh the risks, the drug is safe enough to approve

Current Process at fda.gov

 Pre-Clinical Research  Preliminary safety studies evaluate the drug’s toxic and pharmacologic effects through in vitro and in vivo laboratory animal testing  Develop a pharmacological profile of the drug  Determine the acute toxicity of the drug in at least two species of animals o How much of the drug will it take to kill the animal?  Conduct short-term toxicity studies ranging from 2 weeks to 5 months o Repeated dosage  Median time to completion: 3.5 years Clinical Study – Phase 1  Initial introduction of an investigational new drug into humans  Determine safety and dosage  20-80 healthy human volunteers o Median time to completion: 1 year Clinical Trial – Phase 3

  

Purpose: determine the efficacy of a drug to treat patients with a specific disease, and to learn about common short-term side effects or risks 100 to 500 patient volunteers o Those who have the disease that the drug is intended to cure These are randomized, placebo-controlled, and fully blinded studies o Median time to completion: 2 years

Clinical Trial – Phase 3  Determine effectiveness and safety while attempting to generalize the results to the larger population  Determine appropriate labeling  Enroll between 100 -5000 patient volunteers  Fully blinded, randomized, and controlled experiments o Median time to completion: 3 years Labeling  Indications and usage o When the drug would be used  Dosage and administration o How often? With food? With water?  Contraindications o When the drug should NOT be used  Warnings and precautions o This medication will make you drowsy o Be careful not to operate heavy machinery  Adverse reactions o If something happens, contact your doctor  Drug interactions o Do not take the drug with alcohol, etc.  Use with specific populations o “For ages 3 and under, contact your doctor to see if this drug is best for you” New Drug Application (NDA)  Following the completion of all three phases of clinical trials, an NDA is filed with the FDA o If the data supports a drug’s safety and effectiveness  The NDA must contain all of the scientific information that the company has gathered o Pre-clinical phase - Phase 3  NDAs typically run 100,000 pages or more Final Approval  The decision to approve a new drug for marketing comes down to answering two questions



o 1. Do the results of well-constructed studies provide substantial evidence of its effectiveness? o 2. Do the results show that the product is safe under the explicit conditions of use in the proposed labeling? Median time to completion: 2.5 years

Phase 4  Periodic evaluation of safety information and reported side effects  Continued monitoring of public use  Generally, problems with long-term use will show up here Vaccines  Biological “drugs”  Same basic approval process  The FDA continues to oversee the production of vaccines after the vaccine and the manufacturing processes are approved  If requested by the FDA, manufacturers are required to submit to the FDA the results of their own tests for potency, safety, and purity for each vaccine lot, including the submission of samples of each vaccine lot for testing  Until a vaccine is given to the general population, all potential adverse events cannot be anticipated  Vaccines undergo Phase 4 studies once they are on the public use market  Vaccine Adverse Event Report System (VAERS) to identify problems after public use of vaccines o People can submit the side effects that they have been experiencing o From 1986 until now

POLIO VACCINE Polio Vaccine  The Salk polio vaccine field trials constitute one of the most famous and one of the largest statistical studies ever conducted  The trials are of interest to students of statistics because two different designs were used  Comparing the two designs may provide insight into the relative importance of tightly controlled studies What is Polio?  Poliomyelitis, or infantile paralysis, is an infectious viral disease that enters through the mouth and is usually spread by contaminated drinking water or food  Most healthy people infected with the virus experience little more than mild fever or diarrhea



In about 1% of infections, the virus spreads to the bloodstream and central nervous system, causing varying degrees of paralysis, and in many cases, death o Ex: FDR

1916 Epidemic  New York City - June to August  2,400 dead  9,000 paralyzed  Most were children under 10  Largest polio epidemic ever recorded  Families quarantines to their homes  Children were kept inside; Public events cancelled  Movie theatres banned children under 16  Federal government issued regulation that no child could leave NYC without a certificate of health  Hospitals containing polio patients were burned by vigilante groups  Police in nearby towns guarded entrances to keep NYC residents out  Cats and dogs were killed in overwhelming numbers Polio Cases from 1930 to 1955

 

No (0) cases of Polio have started in the United States since 1975

Salk Vaccine – Pre-Clinical Phase  Animal studies o Monkeys  Three strains of Polio virus  Received $200,000/year from a National Foundation to develop a vaccine Vaccine Protocol  Treatment protocol  Must inactivate the virus – kill it o Safety  Retain enough surface features to create antibody reaction for immunity

o Potency Vaccine Protocol 1  Treat with formaldehyde for 10 days, diluted by filtering every day  Suspend final solution in mineral oil (adjuvant)  Injected first vaccine into residents at the Polk State School, which houses mentally Handicapped boys and young men o Single dose Results and Evaluation  All residents developed antibodies  Salk injected his family  Equivalent to a Phase trial  This vaccine was ultimately rejected because of the mineral oil – it created painful arms and abscesses in a few patients New Vaccine Protocol  Vaccine using monkey kidney cells  Less formaldehyde o Diluted 1:4,000 instead of 1:250  Higher temperatures o 98 F from 33 F o (instead of 37 C from 1 C)  Vaccine treated for 9 days to inactivate  Recommended requiring 11 consecutive lots without problems before considering manufacturing process to be reliable o E.g. live viruses  In 1954, National Foundation on Infantile Paralysis funded the beginnings of a large field trial (Phase 2) Treatment Protocol  55-page protocol for manufacturing the treatment o Detailed descriptions about how to manufacture the vaccine  Treatment fidelity and contamination o Wanted to make sure that the vaccine worked  Type of filter (same)  Number of samples to collect during inactivation process  When and how to test for quality o Testing for quality throughout the whole protocol 1954 Field Trial  420,000 injected with vaccine  200,000 placebo

 

o Injection with saline 1.2 million Natural History o Injected with nothing 3 injections

Design 1  Observed control o Vaccination of second-graders of selected schools in selected areas of the country – with the consent of the children’s parents  The vaccinated second-graders form the treatment group  The first and third graders at the schools would not be given the vaccination, and would form the control group  No randomization o Quasi-experimental design Problems with Observed Control  Selection bias o Who gets the vaccines? o Children with higher incomes  More education about vaccines, more likely to consent for their children  Diagnostic bias o Physicians having to make diagnoses KNEW who had gotten the vaccine and who did not Design 2  First, a sample of children would be selected, all of whose parents consented to vaccination  The sample would be randomly divided into two groups o Random assignment = Increased internal validity  One group would be given the polio vaccination; the other group would be given a placebo o Three injections of saline that would appear identical to the three injections of the real vaccine Two Designs  This type of design is known as a randomized controlled experiment  The randomization tends to nullify all effects (lurking variables) except the treatment effect  None of the participants would know the group identity o Not the child, not the parents, and not the examining doctors  An experiment of this type, in which both the subjects and the evaluators are ignorant of the treatment/control status, is known as a double-blind experiment



The randomized control, double-blind design is considered the golden standard of statistical designs

Iatrogenic Effects  Could the vaccine possibly be a cause of getting polio?  What about the 100+ children who got polio despite receiving the vaccine? Explanations  No vaccine is 100% effective  May have been infected before immunity was conferred  Different strain of polio  Vaccine-caused Evaluation  Small number of cases (of people who got polio after getting the vaccine)  Lack of correlation between injection site and site of paralysis  No association for time between vaccination and occurrence o Suggested that vaccine did not cause the disease 1955 Vaccination  Large scale production using 6 vaccine makers  Minimum requirements – 5 pages  Did not specify that the original Salk protocol (1954) had to be followed Cutter Incident (1955)  Cutter was one of the manufacturer’s  Like others, had trouble making lots that were free of live virus  Using a different protocol that was successful during the efficacy trials  However, only released lots that were measured to be “free” from live virus  

1954 – Required 11 consecutive lots free of live virus 1955 – “A method which was consistently reliable in inactivating the live virus” o No standard for “consistency reliable”

 

1954 – Established that at least 4 reliable samples be obtained during the manufacturing process to indicate that acceptable progress was being made 1955 – Only required that the final lot be free of live virus

 

1954 – Formaldehyde in concentrations of 1:4,000 1955 – Only if formaldehyde was used – another inactivation agent could be used



1954 – Entire manufacturing history was reported (NDA procedure)



1955 – Manufactures were not legally required to report batches of vaccine that failed… as long as it wasn’t sold to the public

Results  1/3 of Cutter’s vaccine lots had failed the final test  Mid-April 1955 – Cases of vaccinated children began to get Polio o All from the Cutter product  Vaccine recalled at the end of April 1955  51 children paralyzed, 5 dead  1/3 had cases of abortive polio o Polio that could have been prevented with a proper vaccine  Plus, polio was then spread to others  Eventually, 220,000 people infected o 70,000 persistent muscle weakness o 164 severely paralyzed o 10 dead What Went Wrong?  Safety tests inadequate o One dose sample was insufficient o Old standard was 1.5 liters  Used too virulent a strain for Type 1 polio virus  Filtration method for removing debris was changed from Seitz filters to glass filters o Thought Seitz filters were too slow  Safety tests on monkeys inadequate o Salk standard required injecting monkeys with steroids to weaken immune system before injecting the vaccine  Let filtered virus sit in refrigerator for long periods before inactivation process began  Never constructed a graph during the manufacturing process to evaluate whether or not the inactivation process was working  Never told anyone they were having problems inactivating the virus  Cutter was never able to make 11 consecutive lots free of the vaccine o It was never able to make 4 consecutive lots  Treatment protocols form the efficacy study were violated Silver Lining  100 million doses were given without incident Broad Concept  Efficacy and effectiveness are different concepts  Efficacy under ideal conditions o How well it works within the lab  Effectiveness in real-world conditions

o How well it works in the real world

Deliberate vs. Unintentional Bias  Do you agree that… (Deliberate)  Pro-choice vs. “baby-killers” (Deliberate) Desire to please vs. Lack of memory  Sometimes not able to remember examples to provide on a survey (Lack of memory)  (DTP) Respond in a way that may conform to the majority o Ex: say that you don’t smoke at all, even though you do sometimes

Placebo Effect Origins of the Placebo  Latin for “I will please”  In Europe, it was customary for the mourning family to distribute nourishment to the congregation immediately following the Vespers of the Dead ritural  Not surprisingly, distant relatives and other unrelated individuals would attend the ceremony, simulating great anguish and grief – in the hope of being given food and drink  This practice was so widespread that these individuals were soon recognized as the personification of all things useless; and were considered to be the prototype for all simulators  Because the grief simulators’ first collective act was to chant “Placebo Domino in regione vivorum”, they were collectively labelled as “Singers of Placebo” o Eventually reduced to just “placebos” or “fakers” Early Medicinal Use  The subsequent reduction of the patient’s symptoms was attributed to the patient’s belief in the drug  Often a doctor who prescribed “tonics” in this fashion would label the fake medicine “Obecalp” Modern Clinical Applications  A 2003 study of Danish general practitioners found that 48% had prescribed a placebo at least 10 times in the past year o Most frequently for viral infections and vitamins for fatigue  Specialists and hospital-based physician reported much lower rates of placebo use  A 2004 study in the British Medical Journal of physicians in Israel found that 60% used placebos in their medical practice



o Most commonly to “fend off” requests for unjustified medications or to calm a patient Of the physicians who reported using placebos, only 15% told their patients they were receiving placebos or non-specific medications

Criticisms  It is unethical to prescribe treatments that don’t work  Telling a patient that a placebo is a real medication is deceptive and harms the doctorpatient relationship in the long run  Using placebos can delay the proper diagnosis and treatment of serious medical conditions Further Uses  About 25% of physicians in both the Danish and the Israeli studies used placebos as a diagnostic tool to determine if a patient’s symptoms were real, or if the patient was malingering  Both the critics and the defenders of the medical use of placebos agrees that this was unethical o “That a patient gets pain relief from a placebo does not imply that the pain is not real or organic in origin” Legitimate Use  Useful in cases where recommended drugs cannot be used  Burn patients who are experiencing respiratory problems that prevent the use of typically prescribed morphine or opioid derivatives  Placebo injections provide “real” pain relief if told they are being given a powerful dose of painkiller Placebos as First-Line Treatments  “Lies that heal”  Between 30 and 40% of any treated group would respond to a placebo  Placebos have been shown to work for certain conditions in approximately 50 or 60% of subjects o Pain o Depression o Heart ailments o Back pain Will it work?  Evidence shows that placebos can work in a clinical setting so long as at least one party believes they are the genuine article  But can a placebo make you better if you know it is a placebo?  Does a placebo cease to be a placebo when you call its bluff?