In its continuing efforts to keep the public informed about the ongoing admissions litigation, the University of Michigan makes these transcripts of the trial proceedings in Grutter v Bollinger, et al., Civil Action No. 97-75928 (E.D. Mich.), available to the University community and general public. As is often the case with transcription, some words or phrases may be misspelled or simply incorrect. The University makes no representation as to the accuracy of the transcripts.
1
1 UNITED STATES OF AMERICA
2 IN THE UNITED STATES DISTRICT COURT
3 FOR THE EASTERN DISTRICT OF MICHIGAN
4 SOUTHERN DIVISION
5 BARBARA GRUTTER,
6 for herself and all others
7 similarly situated,
8 Plaintiff,
9 -vs- Case Number:
10 97-CV-75928
11 LEE BOLLINGER, JEFFREY LEHMAN,
12 DENNIS SHIELDS, and REGENTS OF
13 THE UNIVERSITY OF MICHIGAN,
14 Defendants.
15 -and-
16 KIMBERLY JAMES, et. al.,
17 Intervening Defendants.
18 ______________________________________/ VOLUME II
19 BENCH TRIAL
BEFORE THE HONORABLE BERNARD A. FRIEDMAN
20 United States District Judge
238 U.S. Courthouse & Federal Building
21 231 Lafayette Boulevard West
Detroit, Michigan 48226
22 Wednesday, January 17, 2001
23 APPEARANCES:
24 FOR PLAINTIFF: Kirk O. Kolbo, Esq.
25 R. Lawrence Purdy, Esq.
2
1 APPEARANCES (CONTINUING)
2 FOR DEFENDANTS: John Payton, Esq.
3 Craig Goldblatt, Esq.
4 Stuart Delery, Esq.
5 On behalf of the Defendants
6 Bollinger, et. al.
7
8 George B. Washington, Esq..
9 Miranda K.S. Massie, Esq.
10 On behalf of Intervening Defendants.
11
12 COURT REPORTER: MARY F. WISNESKI, CSR-0231
13 Official Court Reporter
14
15
16 Proceedings recorded by mechanical stenography.
17 Transcript produced by computer-assisted
18 transcription
19
20
21
22
23
24
25
3
1 I N D E X
2 WITNESS PAGE
3 KINLEY LARNTZ, Ph. D.
4 Direct Examination by Mr. Kolbo 7
5 Cross Examination by Mr. Delery 116
6
7
8 E X H I B I T S
9
10 NUMBER IDENTIFICATION ADMITTED
11 137 Dr. Larntz Report 28
12 142 Fifth Supplemental Expert Report 30
13 68 Dr. Larntz Report 30
14 16 1995 Final Grid 93
15 143 Policy 115
16
17
18
19
20
21
22
23
24
25
4
1/17/01 - BENCH TRIAL - VOLUME II
1 Detroit, Michigan
2 January 17, 2001
3 * * *
4 THE COURT: Ms. Massie called. Is she here yet?
5 I was going to say we'll wait for you.
6 MS. MASSIE: Actually we got to the source of the
7 problem just after.
8 THE COURT: Murphy's Law. That always happens to
9 me. I'm in the middle of a jam of the expressway and as
10 soon as I hang up, it's gone and I'm here.
11 MS. MASSIE: Wait a minute, what you are doing.
12 I'll have to try it next time.
13 THE COURT: Everybody here? Let the record
14 reflect, looks like she's here.
15 MR. PAYTON: Your Honor, I just wanted to say
16 something quickly about the --
17 THE COURT: Sure.
18 MR. PAYTON: Chart that I used yesterday that
19 showed the plot of all of the admitted students in 1997,
20 minority and majority.
21 THE COURT: 182, 183?
22 MR. PAYTON: That's correct.
23 THE COURT: Yes.
24 MR. PAYTON: There were questions about whether or
25 not there are LSAT scores at the range of, say, twenty.
5
1/17/01 - BENCH TRIAL - VOLUME II
1 Let me explain just briefly what we did. Although there
2 aren't LSAT scores of twenty or forty, and most of them are
3 120 or 180, in fact, Law Services does report LSAT scores
4 of zero for everyone who takes a non-standard form of the
5 tests; that is persons who are given it by hand because
6 they are disabled. And that shows up as a data point of
7 zero. And that's in the database. So in the database,
8 there are LSAT scores of zero. And they show on our chart,
9 and if you look at the line, it will be scores on the very
10 bottom line of zero, because there's reported like that.
11 So we try to present a chart that could account for all of
12 the data, including the zeros.
13 THE COURT: Okay.
14 MR. PAYTON: Okay. We also wanted to talk about,
15 and I've discussed that with all the parties, what I just
16 said. We also wanted to question something about how we
17 see the rest of our trial days for the next several days.
18 THE COURT: Great.
19 MR. KOLBO: Just on the last point, Your Honor,
20 I'm going to reserve any comments we have on that graphic.
21 And my understanding is Mr. Payton may come here with
22 another graphic as well. And I just wanted to say that for
23 the Court.
24 THE COURT: Okay.
25 MR. PAYTON: Here's, I think, a rough estimate of
6
1/17/01 - BENCH TRIAL - VOLUME II
1 where we're going to go. We believe that we will spend
2 most of today with Mr. Larntz. We believe that tomorrow we
3 will put on President Bollinger, Professor Lempert. We
4 believe that on Friday we will have our Professor
5 Raudenbush and Mr. Shields. And we believe all the rest of
6 our witnesses will probably be done on Monday. We may push
7 over a little bit, but I'm just saying that.
8 THE COURT: Great. No, I appreciate that. And
9 that also gives the Interveners some advance notice to be
10 able to start lining up their witnesses.
11 MR. PAYTON: That's correct.
12 THE COURT: I'll tell you, and I'll say it
13 probably a hundred times. There's nothing like having good
14 lawyers in the case, and that are civil to each other.
15 It's just such a nice way to preside over a case, I'm sure
16 such a nice way for each of you to practice.
17 Generally we have to fight for those kind of
18 things and here you are all agreeing and giving the
19 courtesy to each other to line up witnesses. And as I say,
20 I'll say it a hundred times because that's not enough.
21 It's really a nice way to do it. Okay.
22 MR. KOLBO: Your Honor, Plaintiffs call as our
23 next witness, Dr. Kinley Larntz.
24 THE COURT: Very well.
25 MR. PAYTON: Your Honor, I would also like to
7
1/17/01 - BENCH TRIAL - VOLUME II
1 introduce my partner, Mr. Delery, who will be conducting
2 the examination.
3 THE COURT: Great. Let me have your name one more
4 time.
5 MR. DELERY: Stewart Delery, D-e-l-e-r-y.
6 THE COURT: Thank you.
7 MR. DELERY: You're welcome.
8 THE COURT: Okay.
9 MR. KOLBO: Thank you, Your Honor.
10 K I L N E Y L A R N T Z, Ph. D.
11 was called as a witness and after having been
12 sworn was examined and testified as follows:
13 DIRECT EXAMINATION
14 BY MR. KOLBO:
15 Q. Good morning, Dr. Larntz?
16 A. Good morning.
17 Q. Could you state your full name, please?
18 A. Kinley Larntz. I'll spell it. It's K-i-n-l-e-y
19 Larntz. Last name is spelled, L-a-r-n-t-z.
20 Q. And what is your -- where are you located? Where are
21 you from?
22 A. I'm a statistician. I reside in Minnesota.
23 Q. And are you currently employed?
24 A. I'm currently self-employed.
25 Q. Okay. And how are you -- in the area of statistics?
8
1/17/01 - BENCH TRIAL - VOLUME II
1 A. As a statistician, yes.
2 Q. Okay. Prior to being self-employed, were you employed
3 by others?
4 A. Yes. I was twenty-seven years a faculty member in
5 statistics at the University of Minnesota.
6 Q. Can you tell me, just briefly run through the history
7 of your career at the University of Minnesota, what
8 positions you held there, maybe starting from the beginning
9 and bringing yourself to date.
10 A. Yes. I started at the university in 1971 as a, as we
11 would say, a lowly Assistant Professor, beginning Assistant
12 Professor. I continued, was promoted to Associate
13 Professor in 1977 and promoted to full professor in 1982.
14 And that's the position I held until I retired from the
15 university in 1998. Actually I still maintain a title.
16 It's nice of them to do that. I'm now referred to as
17 Professor Emeritus.
18 Q. Could you describe just briefly your formal
19 educational background?
20 A. Yes. Starting with college, I was an undergraduate
21 math major at Dartmouth, graduated in 1967, and continued
22 my studies as a graduate student at the University of
23 Chicago and did my Ph.D. in 1971 in statistics.
24 Q. Do you have any professional association or
25 memberships that you belong to?
9
1/17/01 - BENCH TRIAL - VOLUME II
1 A. Well, the two I maintain now are, the American
2 Statistical Association is the primary society for
3 statisticians, professional association for statisticians
4 in the country and the American Society of Quality, which
5 is, serves the same function for people interested in
6 issues of quality control.
7 Q. And have you been involved in any publications in your
8 field of expertise?
9 A. I have published. I have to say, I wouldn't have been
10 an Associate Professor or Full Professor if I had not
11 published, that's for sure.
12 Q. What's the principal journal in your area of expertise
13 in statistics?
14 A. Well, there are several journals. I guess Journal of
15 the American Statistical Association is a major journal.
16 I've certainly published there. Journals, I'm just trying
17 to think. There's a whole series been published by
18 different societies.
19 Q. Have you been involved in any editorial positions with
20 any of these statistical journals?
21 A. Yes. Sure. I served as associate editor for the
22 Journal of the American Statistical Association on several
23 occasions and I also served as editor of another journal of
24 the association called the American Statistician.
25 Q. In addition to your academic post at the University of
10
1/17/01 - BENCH TRIAL - VOLUME II
1 Minnesota, have you, in the course of your career, engaged
2 in outside consulting work in the area of statistics?
3 A. Yes.
4 Q. Can you just give us a description of what kind of
5 consulting work you've done over the years?
6 A. Well, consulting work I've done, I have to say, as
7 part of my duties at the university. I was also a
8 consultant within the university, so I worked quite
9 extensively as part of my job collaborating with other
10 researchers on research projects. And I did do some, and I
11 do a bit more now, consult for companies and government,
12 agencies.
13 Q. Can you give us some examples of the government
14 consulting that you've done over the years?
15 A. Okay, sure. I've been, worked with National Science
16 Foundation, National Institutes of Health and extensively
17 with an agency called, the National Institute of Justice,
18 overseeing a series of experiments that were done in police
19 departments on responses to domestic violence. I've worked
20 as a consultant, actually a member of a Scientific Advisory
21 Board to the Environmental Protection Agency concerning
22 small particulates. And, as a say, on the Scientific
23 Advisory Commitment that made recommendations concerning
24 that.
25 My current activity, with respect to government
11
1/17/01 - BENCH TRIAL - VOLUME II
1 consulting is I'm a consultant, statistical advisor, to the
2 Food & Drug Administration, primarily in the area of
3 orthopedic devices. And so I sit on the panel, typically
4 sit on the panel that advises the FDA on whether the new
5 medical device are to be approved or not.
6 Q. Can you give us, your -- I believe you indicated that
7 you, your post at the University of Minnesota has been in
8 the area of applied statistics?
9 A. Well, universities are organized in different ways, as
10 I'm sure everyone knows. And in Minnesota, we had two
11 departments in statistics, or actually several statistic's
12 departments. But one was called Applied Statistics and
13 that was where my appointment was.
14 And I explained that I did internal consulting
15 work within the university, and that was a responsibility
16 of each person in that department to carry on that
17 collaborative research.
18 And applied statistics as it was defined there,
19 and as I generally define it, is the area of using
20 statistics in subject matter fields. So making sure that
21 when you look at data or gather data, design studies to
22 gather data, that they are done statistically
23 appropriately.
24 Q. Can you just give us an idea of what kind of fields
25 one would use applied statistics?
12
1/17/01 - BENCH TRIAL - VOLUME II
1 A. I mean, statistics is useful everywhere.
2 Q. Yes.
3 A. You know that. And I believe that, sir.
4 THE COURT: That's why I went to law school, so I
5 didn't have to take statistics.
6 A. You had no statistics in law school?
7 THE COURT: None. Otherwise I would have probably
8 been in podiatry school or something.
9 A. I've had no law, of courses, either.
10 Q. Let me ask you, Dr. Larntz, for example, in the area
11 of medicine, medical devices, medical cures, is applied
12 statistics used in that area at all?
13 A. Certainly. I use that in that area. I was going to
14 say before that I, I basically am probably too broad in the
15 sense that I've used statistics in lots of areas across the
16 board, starting from the -- well, let's see, I've worked on
17 the, determining the wealth of the United States in 1775.
18 That's a long time ago, and I worked on that.
19 I worked on the composition of fishery by catch by
20 Japanese fishing boats in the Pacific to see what they take
21 into their nets, to see how many tuna they get and how many
22 other things arrive.
23 I've worked, as you say, in medical devices. I do
24 quite a bit of work in that area now. I've worked in
25 engineering. And I've actually written software that's
13
1/17/01 - BENCH TRIAL - VOLUME II
1 used for designing experiments by engineers.
2 Q. Have you done any consulting work in the past, in
3 connection with discrimination cases involving, say,
4 employment or other areas?
5 A. Yes, I have.
6 Q. Were you asked in this case to take a look at certain
7 issues regarding the use of race as a factor in the
8 admissions process?
9 THE COURT: Can I just ask one question? You said
10 that at the university when you applied the statistics
11 department that you were required to do some internal
12 statistics for the university.
13 A. Yes.
14 THE COURT: What kind of statistics did you do for
15 the university?
16 A. Oh, internal. What I did was I worked with
17 researchers in various areas.
18 THE COURT: I see.
19 A. So someone, say, for instance, who was working -- in
20 fact, a large part of my work was with people in medicine.
21 Someone who was getting a grant or wanted to get a grant,
22 study the effect of a drug on HIV, which I worked on for
23 about seven years.
24 THE COURT: But not academic statistics at the
25 university as part of your --
14
1/17/01 - BENCH TRIAL - VOLUME II
1 A. -- Academic statistics in the sense of?
2 THE COURT: I don't know, whatever they may be?
3 A. You mean in the sense of administrative statistics?
4 THE COURT: Yes. Yes.
5 A. That was not what I worked on, per se at the
6 university, no.
7 THE COURT: Okay.
8 A. I was not part of the administration. I was part of
9 a, I'll call it the Research Academic Corps.
10 THE COURT: Okay. Thank you. I'm sorry.
11 Q. Were you asked in this case to take a look at, from a
12 statistical point of view whether, and to the extent to
13 which the University of Michigan Law School takes race into
14 account in the admissions process?
15 A. I was asked to look at the question of examining data
16 for, from the University of Michigan Admissions Office, I
17 presume that's where the data arrived, concerning the role
18 that ethnicity played with respect to admission's
19 decisions, yes.
20 Q. When did you become involved in looking at those
21 issues?
22 A. I believe it was the fall of 1998.
23 Q. And what were you asked to do, specifically?
24 A. Well, I was asked to, if I would be willing to look at
25 the data and see what the data would say concerning the
15
1/17/01 - BENCH TRIAL - VOLUME II
1 role of ethnicity and admissions.
2 Q. Okay.
3 A. And I wanted to, and I took on that job.
4 Q. What information did you consider in, in doing your
5 analysis?
6 A. Well, when I first started, I certainly looked at some
7 materials that were prepared by the law school concerning
8 their admissions policy. The first thing I asked for, you
9 know, being a statistician, the first thing you ask for is,
10 you know, you want to find out what kind of data are
11 available. That's what we, that's our bread and butter.
12 And so I, I would understand that there would be a
13 large number of applicants. So I wanted to try and make
14 sure I got data in a computerized form. And I eventually
15 got computerized data bases of material concerning law
16 school admissions.
17 Q. Okay. Did you look at any documents with respect to
18 the law school admissions?
19 A. I certainly looked at some documents. I think I
20 looked at some, there's certainly a version of an
21 admission's policy. There was, I think a visitor's report
22 that discussed the law school and some written materials,
23 some tabulations presented by the law school.
24 Q. I think you may have up there with you a free-standing
25 copy of Exhibit 4. Do you see that? Is there such a
16
1/17/01 - BENCH TRIAL - VOLUME II
1 document up there?
2 A. Yes.
3 Q. Can you just take a look at that and tell me if that's
4 one of the documents you took a look at as part of
5 undertaking your analysis?
6 A. Yes. I've looked at this document before.
7 Q. Okay. And was that represented to you to be the law
8 school admission's policy that was in effect for the years
9 that you were going to take a look at the data?
10 A. Well, it says it's admissions policies. That's what
11 it says on here. It's dated 4/22/92, and that predates the
12 database dates. I don't, I don't know that there's another
13 admission's policy after that.
14 Q. That's one of the documents you looked at?
15 A. Yes.
16 Q. In addressing the issues you looked at?
17 A. Yes.
18 Q. How many years of -- you mentioned, I think that you
19 obtained in electronic form the data from the law school,
20 certain data from the law school?
21 A. Yes. Over a period of time, I received large
22 databases, yes.
23 Q. And how many, how many years of law school data did
24 you look at?
25 A. I was given datas, initially from 19, they covered the
17
1/17/01 - BENCH TRIAL - VOLUME II
1 admission years of 1995 through '98. And then subsequently
2 received 1999 admission's data and the year 2000
3 admission's data.
4 Q. Okay. And in what form, you mentioned that it was
5 electronic. Can you just describe the format in which the
6 data was supplied to you?
7 A. It was a Microsoft access database.
8 Q. Okay. And what kinds of data were included in the
9 database? What kinds of information, generally speaking,
10 was contained in the database?
11 A. Well, there were a large number of tables in that
12 database. And they contained information, for instance, on
13 ethnicity, which given the subject of this case, that would
14 be important to have. Information on credentials, grade
15 point average, admissions test score. And there was an
16 index, maybe, I call it a selection index, information in
17 coded form on what school people went to as undergraduates,
18 a variety of information of that sort.
19 Q. Okay. And was the data provided to you in
20 substantially the same form for each of the years that you
21 were provided data?
22 A. Essentially. I mean, there's probably some small
23 changes in coding, but, essentially, the same form, for
24 which I admit I was grateful, so I didn't have to change
25 everything each year.
18
1/17/01 - BENCH TRIAL - VOLUME II
1 Q. Okay. Can you describe just generally, in a very
2 general fashion to begin with, what types of statistical
3 analysis you performed with respect to the data that you
4 received, just very generally.
5 A. In general the terms I -- I used some descriptive
6 statistics to look at the characteristics of the
7 individuals in the database, the applicants, and, and then
8 I also then used, I guess, in general terms, I used
9 statistical methods to examine the odds or chance of
10 admission as a function of credentials of the applicants.
11 And I guess in statistics when we get fancy, we'll call
12 that inferential statistics, but first we call it
13 descriptive statistics.
14 Q. Okay. We'll get into the details then of your
15 conclusions. We'll go through a number of items. But did
16 you form some overall conclusions? You stated some general
17 fashion of some overall conclusions that you came to with
18 respect to your analysis of these databases?
19 A. Yes.
20 Q. And were those conclusions formed to a reasonable
21 degree of certainty in the field in which you practice?
22 A. Oh, I feel quite comfortable with my conclusions, yes.
23 Q. Okay. Can you just generally describe, in general
24 fashion, what conclusions you, you drew from the data that
25 you reviewed?
19
1/17/01 - BENCH TRIAL - VOLUME II
1 A. From the data I reviewed, which is the description, in
2 trying to describe as best I could, as it was done, the
3 admissions that were done by the University of Michigan Law
4 School. And with respect to ethnicity, I found, this is
5 not a statistically technical term, but I found an
6 incredibly large allowance given to members of selected
7 minority groups with respect to their chance of admission.
8 And that all of my analyses that I did confirmed and
9 continued to confirm that for individuals that have similar
10 credentials.
11 Q. And was that true for all the years you looked at, or
12 just some of the years or?
13 A. The basic substantive conclusion. The basic
14 substantive conclusion. Of course the numbers are going to
15 vary from year to year. And if they didn't, I wouldn't be
16 in business, I guess, as a statistician. But the numbers
17 varied a fair bit. But the actual substantive conclusions
18 were exactly the same across over the years, yes.
19 Q. Now, you mentioned before that you did different types
20 of analysis. And I just want to spend a little time on
21 some detail there. You mentioned descriptive statistics.
22 What does that refer to, descriptive statistics?
23 A. Well, it describes the data, how's that? What it does
24 is try to give some summary numbers, and in a couple cases
25 I looked at summary pictures of the data to try to
20
1/17/01 - BENCH TRIAL - VOLUME II
1 understand the characteristics of the applicants, and in
2 particular, the characteristics of applicants who were
3 admitted to the law school.
4 Q. And then you've mentioned that you also employed
5 inferential statistics in your analysis?
6 A. Well, I employed statistics that would allow us to, at
7 least, when I say draw conclusions, as to whether or not
8 the, the things we were seeing are just due to chance or
9 not. And that's what I mean by inferential statistics.
10 And I certainly, the main technique I used in that
11 was a technique called logistic regression, which is a
12 standard technique that we use for looking at a binary
13 response, binary in the sense of admit or not admit, to the
14 law school, and analyzing what, what would effect that,
15 that particular response and particular, the relation of
16 that to grades, undergraduate grades, admissions test
17 scores and other factors.
18 Q. Is an examination of relative odds or odds ratios, is
19 that a form of inferential statistical analysis?
20 A. Well, it is, and logistic regression specifically
21 looks at trying to understand odds as the response, the
22 odds of, say, odds in this case, odds of admission, yes.
23 Q. So relative odds analysis is related to an analysis
24 using a logistic regression?
25 A. Logistic regression, actually the technique itself
21
1/17/01 - BENCH TRIAL - VOLUME II
1 analyzes its response, technically, the log rhythm of the
2 odds. And then when you're doing comparisons between one
3 odds and another odds, and you'll get to that, I presume.
4 We would calculate what I call relative odds or odds
5 ratios.
6 My students always accuse us in statistics of
7 having one concept in fourteen names. It makes taking
8 exams difficult. And odds, and relativity odds and odds
9 ratios are the same, the same thing.
10 Q. Can you just then define for us what in the area of
11 statistics what odds and relative odds mean?
12 A. I think, yeah, it's probably a good idea, because odds
13 is used a lot in, in, well, in our everyday life, so I can
14 do that. Is it possible I can write on the board a little
15 bit?
16 MR. KOLBO: With the Court's permission.
17 THE COURT: I have no problem. The only problem
18 it's going to kind of screw up your Elmo.
19 MR. KOLBO: I think we're actually going to use
20 this over.
21 (Whereupon an off-the-record
22 discussion was had.)
23 Q. Do you have a pen there, Dr. Larntz?
24 A. Okay. Well, what I want to do is just define for you
25 odds. And I know -- I don't presume that you know a lot
22
1/17/01 - BENCH TRIAL - VOLUME II
1 about odds. And I'll just, I'll just.
2 THE COURT: Just in Vegas.
3 A. Well then, I particularly, then I particularly need to
4 define what I mean. And in statistics we talk of the odds
5 in favor of an event.
6 And just to be clear, most odds, as reported in
7 Vegas, and I have to say the odds aren't always, you know,
8 always, they're always put in the form of odds against,
9 typically. And so, and I'll leave my comments out about
10 the Vikings.
11 But we think of the odds as a number, so in
12 statistics, we think of odds as a number. So, for
13 instance, if we have what we think of as fifty, fifty odds,
14 50/50, odds of 50/50, that means that's like a coin flip.
15 THE COURT: If you guys want to sit in the jury
16 box, you're more than welcomed to, or you may stand.
17 Whatever makes you happier.
18 A. I don't think we'll be here forever. And what we do
19 in statistics is we take the ratio of these. So fifty
20 divided by fifty. That is the chance, the chance against,
21 and we divide these. And we call, we make that a number,
22 okay. And so that would be, in our terms, an odds of one.
23 Okay. So what do we do for other chances, or probabilities
24 of admissions? Suppose there's a 25 percent chance; how
25 would we convert that to odds, just so we're clear.
23
1/17/01 - BENCH TRIAL - VOLUME II
1 So that would be what, twenty-five for to
2 seventy-five against. We do the same thing, make it a
3 ratio and then we get twenty-five over seventy-five, and
4 that's a number. The number is one third.
5 Similarly, if we did 75/25, again, whoops, 75/25,
6 that would be a number. And we get the number,
7 seventy-five divided by twenty-five, we get three. So
8 those are, those are how we calculate odds. We can do the
9 same thing for others. And I'll probably take another
10 example in a second.
11 Now, when we're comparing odds, the odds of two
12 events, and that's what we're going to do here a fair bit
13 is compare odds. We take what's call the relative odds or
14 the odds ratio. And the odds ratio then for, say, the
15 event that has probability 75 percent, to the event that
16 has probability of 25 percent, the odds ratio is just the
17 ratio of the odds. And so the odds ratio then would be
18 three divided by one third, and -- and we've divided by
19 fractions at one time in our lifes, yes.
20 THE COURT: I did.
21 A. Okay. And we come up with nine. That's the odds
22 ratio. Take one more example.
23 Suppose the, so an odds ratio of nine corresponds
24 to comparing seventy-five to twenty-five. Another one,
25 say, ten to ninety would give us, what, I think I can skip
24
1/17/01 - BENCH TRIAL - VOLUME II
1 this and just write the odds as one ninth, ten to ninety,
2 say, ninety to ten. That gives is an odds of nine. And if
3 you're comparing an event that has probably 90 percent to
4 an event with the probability of ten percent, we
5 calculation an odds radio, odds ratio, then of nine divided
6 by one ninth, which would be equal to eighty-one. So the
7 concept of odds, essential to comparison of two events,
8 probabilities are odds ratio.
9 Q. Doctor, how would one communicate that in sort of a
10 sentence if one wanted to express the relative odds of a
11 ten percent probability of something occurring, versus a 90
12 percent probability of something occurring?
13 A. In statistical terms we'd say that the odds ratio;
14 that is, the odds of, the odds ratio of the event, whatever
15 we call it, ninety, that has 90 percent probability to the
16 percent that has ten percent probability has an odds ratio
17 of eighty-one.
18 Q. Very good. Do you want to resume, resume the witness
19 stand. And let me ask you, in the area of statistics, does
20 it happen sometimes that matters simply occur by chance?
21 A. Does it happen?
22 Q. Yes.
23 A. In statistics, I think in the real world it happens
24 that things occur by chance.
25 Q. Okay. And in your profession, are there ways in which
25
1/17/01 - BENCH TRIAL - VOLUME II
1 one can summarize in which statisticians summarize the
2 degree to which an observed difference, say, a ten percent
3 probability versus a 90 percent probability might simply be
4 due to chance, as opposed to something else?
5 A. Yes. We often summarize, if we're doing statistical
6 comparison, we want to understand whether or not the events
7 mean, anything can happen. So do the events that we see,
8 are they due to chance alone or not. And we summarize that
9 several ways, quite commonly use what's called a P value to
10 summarize the degree of evidence concerning whether or not
11 something occurred by chance.
12 Q. Is the term, standard deviation, used as well in that
13 context?
14 A. Lots of ways of -- remember in statistics there's
15 often several ways to do the same thing. And so what we do
16 is we have the P value, may be a summary, or we may
17 summarize also in terms of standard deviation. So, for
18 instance, and are we going to use the board a lot? I'm not
19 sure whether these people need to stay up there or not?
20 It's up to you, Your Honor.
21 THE COURT: I think they like it there. They can
22 move whenever they want.
23 A. Okay. I'm going to take a drink of water because I
24 was counting on that when they moved.
25 THE COURT: Help yourself.
26
1/17/01 - BENCH TRIAL - VOLUME II
1 Q. Could you just summarize what standard deviation
2 analysis tells a statistician, just kind of summarize just
3 what kind of values you look at for statistical
4 significance?
5 A. What I think I better do is talk about P value and
6 standard deviation both, and we summarize the, we
7 calculate, if we're describing an outcome, whatever that
8 outcome is. We calculate the probability that we see that
9 outcome or a more significant outcome than that, if chance
10 alone were operating.
11 And in statistics we often talk of P values. We
12 have magic numbers -- I shouldn't say that too loud, there
13 might be other statisticians listening. But magic numbers
14 of .05, five percent.
15 So if an event has a five percent or less chance
16 of occurring under chance, we usually call that
17 statistically significant. That's a term that arises. We
18 get, people like us to summarize succinctly, so if it has
19 something, an event has a one percent chance or less, we'd
20 often call that highly statistically significant. And I
21 think we've stopped at that particular point.
22 Now, statistics that we look at we can often
23 summarize the departure from chance in terms of numbers of
24 standard deviations. And this is also done quite often
25 with lots of summary statistics. And so many of the tests
27
1/17/01 - BENCH TRIAL - VOLUME II
1 we do are summarized in terms of a statistic Z score, which
2 gives the number of standard deviations that the outcome is
3 away from, the chance alone outcome.
4 A two standard -- well, I should say, I'll be
5 precise, 1.96 standard deviation corresponds to a five
6 percent P value, so 1.96, we get a little sloppy. We say
7 two, okay, in statistics. And so we'll say about two
8 standard deviations corresponds to an event that's
9 statistically significant.
10 And if we summarize in terms of standard
11 deviations, a one percent event is, see if I remember the
12 number, I once forgot this number in court so I better
13 remember it today. The one percent value is 2.576 and so
14 2.57, 2.58 would correspond to a significance of one
15 percent. So, in general, in statistical terms, events that
16 have, now I'll say it in more summary forms, events that
17 correspond to departures that correspond to two or three
18 standard deviations are generally considered statistically
19 important. Events that have, that further departure are
20 obviously much more statistically significant than, than
21 that.
22 Q. For example, what would a standard deviation of five
23 signify?
24 A. Five?
25 Q. Yes.
28
1/17/01 - BENCH TRIAL - VOLUME II
1 A. Well, five is, and this is actually literally true for
2 most normal tables, it's off the chart in the sense that
3 its, we believe events that would correspond to a five
4 standard deviations have very, very minuscule chance of
5 occurring by chance alone. Calculation -- I don't have
6 that calculation with me, probably less than one in a
7 million is the probability associated with five or more.
8 Q. Now, I think you've indicated that you wrote some
9 reports in connection with the work that you did in this
10 case?
11 A. One of the things that this witness did was write
12 reports, that's true.
13 Q. Can you take a look, or Dwayne, can you show the
14 witness the books that have Exhibit 137 through, I think
15 it's 142. When you get to 137, can you let me know, Dr.
16 Larntz?
17 A. Okay. I am at 137.
18 Q. Yes. Is that a copy of your report?
19 A. It appears to be. It's dated December 14, 1998,
20 expert report of Kinley Larntz.
21 MR. KOLBO: I'd offer Exhibit 137 at this time,
22 Your Honor.
23 THE COURT: Any objection?
24 MR. DELERY: No objection, Your Honor.
25 THE COURT: Received.
29
1/17/01 - BENCH TRIAL - VOLUME II
1 Q. Will you take a look at 138 to 142 and just tell me,
2 we can speed things up a little bit if you can confirm to
3 me that those are copies of all your remaining reports
4 generated in this case?
5 A. One thirty eight, 139, 140, 141, yes. Those are the
6 additional reports that I entered in.
7 Q. Okay. And then if you could take a look at exhibit,
8 and this will be in a separate book, I think, at this
9 point, Exhibit 68.
10 A. Yes, I see Exhibit 68.
11 Q. Is that data that you assembled from the database that
12 you reviewed in connection with this case?
13 A. Yes. These are spreadsheets that I, that summarize
14 the data that was in the databases for 1995 through '98
15 concerning law school admissions, yes.
16 MR. KOLBO: Maybe to get things in order here?
17 Your Honor, I will at this time offer Dr. Larntz as an
18 expert witness in this indication. I have to do that?
19 THE COURT: Any voir dire? Anybody have any
20 objection to him testifying?
21 MR. DELERY: No, Your Honor.
22 MS. MASSIE: No.
23 THE COURT: Very well.
24 MR. KOLBO: I don't think I've then offered then
25 exhibits -- it's just been pointed out to me, I'm not sure
30
1/17/01 - BENCH TRIAL - VOLUME II
1 that it makes any difference for our purposes, but 137 also
2 contains Exhibit 1 -- or 137 also contains Exhibit 68, so
3 there's some duplication there.
4 A. Yes. Yes. I actually saw that as I went through.
5 MR. KOLBO: Okay. At this time, then, I would
6 offer the remaining of Dr. Larntz' reports, Exhibits 138
7 through 142, as well as, I guess, Exhibit 68, as long as
8 it's already marked.
9 THE COURT: Any objection?
10 MR. DELERY: No, Your Honor.
11 MS. MASSIE: No Your Honor.
12 THE COURT: Received.
13 Q. Dr. Larntz, in addition to the written reports we have
14 here, have you assembled something in the nature of a video
15 presentation that will help explain your conclusions and
16 analysis that you performed?
17 A. We prepared, picked out some slides which are, for the
18 most part, with a couple of exceptions, copies of the
19 tables and figures from the reports, yes.
20 Q. Let's take a look at the first slide, if we can. Can
21 you tell us, first of all what table one represents? Let
22 me back up a little bit. We're going to go through a
23 number of these slides. Can you tell us, for the most
24 part, where these slides came from? Are they derived from
25 some other records?
31
1/17/01 - BENCH TRIAL - VOLUME II
1 A. This is table one from the first report that I did, so
2 this is from the first, December, 1998 report, table one.
3 Q. For the most part, these, these tables, although maybe
4 in slightly different form, are, in terms of the graphics,
5 the way they're graphically displayed, these are contained
6 in your various reports?
7 A. Exactly, yes.
8 Q. As we go through these?
9 A. Exactly.
10 MS. MASSIE: Mr. Kolbo, can I stop you there for a
11 second? This was not dropped off at our office. And I
12 haven't objected until now because I understood that it was
13 exactly the same as the materials contained in Dr. Larntz'
14 report. If it's not, I'd like a copy so that I can look at
15 what's different.
16 MR. KOLBO: Your Honor, this is an exhibit that
17 was actually delivered with the witness books. It's
18 exhibit what, Dwayne?
19 MS. MASSIE: 143.
20 MR. KOLBO: 143.
21 MS. MASSIE: It was not in the materials that we
22 received. If I can just a have a few minutes to look at
23 it? I assume it's substantially the same?
24 MR. KOLBO: It should have been. I apologize if
25 it wasn't.
32
1/17/01 - BENCH TRIAL - VOLUME II
1 THE COURT: Okay. It's 143 in the book.
2 MR. KOLBO: One forty three in the book.
3 THE COURT: Take your time. Take a look at it.
4 MR. KOLBO: You have it there.
5 MS. MASSIE: We do have it. I apologize.
6 MR. KOLBO: We have an extra copy, Miss Massie, if
7 you'd like.
8 MS. MASSIE: No. You can go ahead. I apologize.
9 Q. Can you tell us, Dr. Larntz, what, what table one
10 summarizes or represents?
11 A. Yes. This first line, which is table one from the
12 first report that I did summarizes by ethnic group
13 classified in exactly the same manner as the variable
14 ethnicity was classified in the law school database.
15 It summarizes by ethnic group the number of
16 applicants for law school for the years 1995 through 1998.
17 So we can see there are about, well, you can see exactly if
18 we read, 4,147 applicants in 1995, and similarly 3,500,
19 about 3,537 applicants in 1998. And by each ethnic group,
20 we can count the number of applicants, so.
21 Q. Dr. Larntz, do you have a pointer there if you want to
22 use one? I'm not suggesting that you have to, but if it's
23 going to be helpful to you to make specific points?
24 A. I do have one, yes.
25 Q. Okay. Can you tell me what, what, if any, general
33
1/17/01 - BENCH TRIAL - VOLUME II
1 conclusions you drew from just table one here?
2 A. Well, these are the numbers of applicants, the raw
3 data. The conclusions are, and by the way, the ordering is
4 exactly in the same order as the variable was in the law
5 school database. I just used that, so if someone asks
6 where did the order come from, that's the order that was
7 used in the code. It just gives the number of applicants.
8 So for instance in 1995 there were 4,147
9 applicants, 45 of the applicants were listed as Native
10 American by ethnicity. 404 African-American, 2,316
11 Caucasian American, 98 listed as Mexican American, 115 as
12 other Hispanic American, Asian Pacific Island American,
13 470, not too dissimilar from the number of
14 African-Americans applicants, Puerto Rican, twenty, small
15 number, foreign, classified as 412, and unknown, 567. So
16 the largest groups with caucasian American, Asian Pacific
17 Island, African-Americans, well, except for the unknowns.
18 Q. Okay. Let's go to the next slide.
19 A. That's similar for all the years, and I think we don't
20 have a prepared a slide for 1999 or 2000, but there were
21 about 3,400 applicants in each of those years and there's a
22 similar distribution.
23 Q. Let's take a look at the next slide.
24 A. Yeah.
25 Q. Can you summarize what this, what information this
34
1/17/01 - BENCH TRIAL - VOLUME II
1 slide provides.
2 A. This is part of the descriptive statistics that I was
3 preparing. And so what I've done here is for the applicant
4 group as a whole; that is. For all applicants, I've
5 summarized the median undergraduate grade point average.
6 So, for instance, in 1995 the overall median was
7 3.49 and the median for Native Americans was 3.14, somewhat
8 lower. These are among applicants, 3.03 for
9 African-Americans, 3.55 for Caucasian Americans, compared
10 to 3.49. 3.31 for Mexican American applicants. 3.56 for
11 other Hispanic American, Asian Pacific Island applicants
12 3.48, 3.14 for Puerto Rican applicants 3.46 for foreign
13 applicants, 3.53 for those unknown ethnicity. And so, and
14 you can see that we've given that, those numbers for 1995,
15 '96, '97, '98, and I think a couple more slides will give
16 '99 and 2000.
17 Q. Let's go to the next slide then.
18 A. And similarly, let's just make sure we see what we're
19 seeing. The median, the median, I guess given the way
20 we're doing things I better make sure I define what a
21 median is.
22 A median is the value in which half the applicants
23 are at or below that value and half are at that value or
24 above. So there are applicants that are higher, higher and
25 lower, but it's the one that splits.
35
1/17/01 - BENCH TRIAL - VOLUME II
1 So, in fact, in 1999 we have actually similar
2 patterns. What we see is with respect to the overall
3 median of 3.52, Caucasian American applicants are, are at
4 3.57, Asian Pacific Island, 3.46, and then Native
5 Americans, 3.37. Probably -- I'm not trying to do this
6 particularly in order, but I'll just point out
7 African-Americans are at 3.15, somewhat lower than, with
8 the median somewhat lower than the overall. Mexican
9 American applicants, 3.36 and Puerto Rican 3.20.
10 Q. Let's go to the next slide down. Is this just for the
11 same information for the next year?
12 A. Exactly. With the same pattern of median.
13 Q. All right. Next slide please, Dwayne. What does that
14 table three summarize there?
15 A. I did the same analysis, looking at LSAT score in
16 calculating the median. These are for applicants as they
17 presented to the, to the law school. And so the median,
18 the median overall for 1995 was 162. And we can see that
19 Native Americans applicants that year had a median that was
20 eight points lower at 154, African-Americans applicants had
21 a median that was 150, twelve points lower, Caucasian
22 Americans applicant had a median that was one point higher,
23 163. Mexican American applicants had a median LSAT of 155,
24 seven points lower than the overall median. Hispanic,
25 other Hispanic American applicants had a median of 156, six
36
1/17/01 - BENCH TRIAL - VOLUME II
1 points lower than the overall. Asian Pacific Island
2 Americans had 161, which was within one point of the
3 overall, Puerto Rican applicants, 155, seven points lower,
4 and you can see foreign and other unknowns also their
5 values down there.
6 Q. And without going into detail you've reported the same
7 information for '98, '96 and '98, correct?
8 A. In '96, 1997, '98 in this, showing, showing similar
9 patterns, yes.
10 Q. Let's go to the next slide down.
11 A. And this is the same, the same display for 1999.
12 Q. Okay.
13 A. And if we look at the next slide, we get the same
14 display for the year, the year 2000.
15 Q. What was the next step in your analysis then?
16 A. Well, well, I just summarized this. I mean, in some
17 sense, the summary of this, we saw, we see that the, a
18 number of ethnic groups had median scores that are lower
19 than the average, consistently across there, and that's
20 among, among the applicants. I just let that be a summary.
21 The next step was, and we're first looking at all
22 applicants, and this is, in some sense, who presented
23 themselves to the law school for admission. Those are the
24 applicants. The next step we looked at is what are the
25 characteristics of the individuals they selected; that is
37
1/17/01 - BENCH TRIAL - VOLUME II
1 the accepted applicants. So we looked at, I looked at,
2 prepared exactly identical tables for accepted applicants.
3 That's the next step.
4 Q. Should we go to the next slide then?
5 A. Sure.
6 Q. And what information is, is summarized just briefly,
7 what information is contained here?
8 A. Sure. This is, I'm sorry. This is the same display,
9 using the median, comparing the actual accepted applicant
10 GPAs. And we can see, for instance, in 1995 the, among the
11 accepted applicants, the average, or not the average,
12 excuse me, the median. The median is a form of average, so
13 I better -- but the median was 3.64, and the Native
14 Americans were at 3.36, somewhat lower, African-Americans
15 were 3.33, among accepted applicants, again, somewhat
16 lower. Caucasian Americans were 3.68, slightly higher.
17 Mexican American applicants were 3.50, and Asian Pacific
18 Island applicants were 3.6, again, within, about a
19 hundredth of a point of the overall and Puerto Rican
20 applicants were 3.3 point, somewhat lower.
21 And so what we see higher, and we see throughout
22 this is that among accepted applicants, Caucasian and
23 American and Asians and non-American applicants had median
24 undergraduate GPAs that were similar or slightly higher
25 than the overall median.
38
1/17/01 - BENCH TRIAL - VOLUME II
1 And applicants from ethnic groups of Native
2 American, African-American, Mexican American, and Puerto
3 Rican, ethnic groups had averages, had undergraduate,
4 median undergraduate GPAs that were somewhat lower.
5 Q. Did you do similar analysis for the later years and
6 have similar conclusions?
7 A. For 1999 and 2000 we did exactly the same thing, yes.
8 Q. Can we have that next slide?
9 A. And that's a report in 1999. And the next slide
10 reports 2000.
11 Q. What was -- let's go to the next slide, Dwayne.
12 A. And then we continued in the same summary manner that
13 we did before for LSAT scores. And LSAT scores for 1995,
14 the overall median was 168. In fact, the overall median
15 was 168 for all four years, for all accepted applicants for
16 '95, '96, '97 and '98.
17 Native American applicants, or accepted
18 applicants, Native American accepted applicants were six
19 points lower the first year, and seven points the second,
20 and seven the third, and eight the fourth year, lower than
21 the overall median.
22 African-Americans accepted applicants were nine
23 points lower the first year, nine points lower the second
24 year, eight points lower the third year and nine points
25 lower the, in 1998.
39
1/17/01 - BENCH TRIAL - VOLUME II
1 Caucasian American applicants were all, either at
2 or one point above their median scores were at or one point
3 above. Mexican American applicants, accepted applicants
4 were eight points lower in '95, five in '96, seven in '97,
5 and eight in '98.
6 Asian Pacific Island American applicants were at
7 or one point above the median for all the years given here,
8 and Puerto Rican applicants, accepted applicants were nine
9 points lower in '95, eight points lower in '96, four points
10 lower in '97 and seven points lower in '98.
11 Q. Let's go, did you do the same analysis then for the
12 later two years again?
13 A. Yes. We have 1999 and we have 2000.
14 Q. Slides twelve and thirteen are for 1999 and 2000?
15 A. That's correct.
16 Q. The same LSAT analysis for accepted applicants?
17 A. The same, reporting the median of accepted applicants,
18 yes.
19 Q. All right. Let's go to the next slide, Dwayne. Can
20 you tell us?
21 THE COURT: What's the difference between "mean"
22 and "median"?
23 A. The median is exactly the value at which half or at
24 that value or above and half or that value or lower.
25 That's the median. Splits the day in half.
40
1/17/01 - BENCH TRIAL - VOLUME II
1 THE COURT: Right.
2 A. We also in percentile terms, that's the fiftieth
3 percentile.
4 THE COURT: Okay.
5 A. The arithmetic mean, the average, if you want to call
6 it that, basically what it does is it sums up all the
7 values and divides by how many you have. Okay. So the
8 average, the arithmetic, we typically call the mean, is, is
9 precisely that calculation. It's just another way of
10 summarizing. We could have done the same thing with
11 averages.
12 THE COURT: How, if you use "mean", how would the
13 charts differ? Other than numbers, would the spread be any
14 different, or how would it look if you used "mean" instead
15 of "median"?
16 (Whereupon an off-the-record
17 discussion was had.)
18 A. Well, I don't recall the exact numbers, Your Honor. I
19 would -- my feeling is it would be similar in, in
20 difference, but I don't. I don't have the actual numbers.
21 THE COURT: I see. That is would probability, if
22 you chart it, it would be about the same?
23 A. In the case what would be different would be if there
24 are extreme values and extreme values would tend to pull
25 the mean up or down, depending on how extreme the value is.
41
1/17/01 - BENCH TRIAL - VOLUME II
1 May I give an example?
2 THE COURT: Sure.
3 A. Not in this area.
4 THE COURT: Yeah.
5 A. The example I used to use in class, I used to use an
6 example of, and I'm try to make it so I can do the math in
7 my head although I've got a conductor here if I have to.
8 THE COURT: Sure.
9 A. So my wife worked in a dental office, okay, and we
10 looked, I looked at the salaries. I'm making up this
11 example, by the way. She did work in the dental office but
12 I'm making up the numbers.
13 THE COURT: Yeah.
14 A. And the individuals working at that dental office, the
15 salaries of the individuals working in the dental office
16 were $20,000 a year, $20,000 a year and $260,00 a year.
17 There were three individuals working in this dental office.
18 Do you see what I'm saying?
19 THE COURT: Uhm-uhm.
20 A. Two at twenty, and two sixty, okay. The arithmetic
21 mean of that is a hundred thousand dollars a year. So
22 that's the average salary in a dental office. And someone
23 might say, that doesn't really summarize what's going on in
24 that office with respect to salary. The median actually in
25 this case would be $20,000, so.
42
1/17/01 - BENCH TRIAL - VOLUME II
1 THE COURT: I see. And why, you decided to use
2 the median here instead of the mean?
3 A. Well, I probably could have used either in this case,
4 but when there are extreme values, the median is not
5 effected by those extreme values.
6 THE COURT: I see.
7 A. And that's, generally, why I would report the median
8 typically, although I don't think there's any evidence in
9 this case that the mean would give any different answer,
10 although I don't have those numbers calculated in front of
11 me.
12 THE COURT: But because there's no extremes like
13 in your example, the median you thought was the way to do
14 it?
15 A. That's what I thought would be the way to do it. You
16 know, there are some reported zeros in the GPAs and the
17 LSAT. I think you actually heard about those earlier, and
18 the median would be effected very little by, if we
19 concluded or excluded those values, and I can't even
20 actually remember whether I did include or exclude those.
21 The mean obviously would be effected quite a lot. It would
22 lower those values quite a bit.
23 THE COURT: Okay. You've answered my question. I
24 appreciate it.
25 Q. Can you tell us in what kinds of analysis you
43
1/17/01 - BENCH TRIAL - VOLUME II
1 performed that you displayed with respect to slide fourteen
2 on your presentation?
3 A. Well, the median isn't the whole story. I wanted to
4 look at the actual distribution of scores and this is, this
5 is a standard summary technique that is used in statistics,
6 part of many statistical passages. A summary that consists
7 of, there are a lot of applicants, let me just say. There
8 are a lot of applicant.
9 And so displaying every single value isn't very
10 useful. And so we need to display a summary, and so what
11 we've done here is display the distribution of the
12 undergraduate grade point averages for accepted applicants
13 and we're displaying this as a function of ethnicity. So
14 let me just try to tell you what this box plot means.
15 What we have here is the, for Native Americans
16 accepted applicants, we have a box and some lines and
17 brackets and another line below, and the values that are
18 highlighted here, if I might, the white space in the
19 middle, that corresponds to the median, okay. So exactly
20 the same values that we have before, the median here would
21 be that white space in the middle of the box. The box
22 extends up to, remember the median was the fiftieth
23 percentile. And what we did is take the value that 75
24 percent of the applicants at or below the seventy-fifth
25 percentile so the box extends out the seventy-fifth
44
1/17/01 - BENCH TRIAL - VOLUME II
1 percentile on the lower and the upper side and the
2 twenty-fifth percentile on the lower side.
3 And then this technique, this box, there's several
4 ways to make them, but what it does, it doesn't report any
5 other values out beyond that unless the values are far away
6 from the box in a relative sense. So what it does is it
7 takes up the other values, the other, let's see there's 75
8 percent here, so about 25 percent of the data up here, so
9 about 25 percent of the accepted applicants in this range
10 here, about 25 percent are in the range below, down to the
11 bracket, unless a value is considered more extreme,
12 relative to its own box. And so there is one value down
13 here, see a bar down here, a line. That corresponds to the
14 undergraduate GPAs of one accepted applicant that's found
15 at that value.
16 Q. Is that one bar down there just representing one
17 applicant?
18 A. I believe it does. It actually could, given the way
19 the computer works. If there were two that had exactly the
20 same score, it would override them. That's true of all the
21 plots that you see, Your Honor. Further sometimes if
22 they're exactly the same values, they're going to overprint
23 unless we do some special, and I didn't do anything to do
24 that.
25 Q. What does the term, outlier, mean in statistics?
45
1/17/01 - BENCH TRIAL - VOLUME II
1 A. Well, there are whole books written on that, none of
2 which I've written, by the way. Outlier means a value
3 that's considered far away from the normal pattern or range
4 of the data. And this particular, this particular figure
5 is constructed so that these values that are printed
6 outside are termed outliers. That's the term that's used.
7 So they're further away from the bulk of the data than the
8 rest of the values.
9 Q. The lines outside the two brackets at the top and
10 bottom are outliers?
11 A. Well, yes, that's what this plot refers them as, yes,
12 outliers, yes.
13 Q. I'll ask you just summarize what your conclusions were
14 based on the analysis you did here for the median GPAs for
15 applicants accepted in 1995?
16 A. Yeah. The GPAs from the plots here, you can see that
17 in looking at these box plots, you can see that the Native
18 Americans box, the African-American box that, they're in
19 the same order as we had for ethnic groups before. And to
20 some extent the Mexican American box, that is the box
21 contains the, between the seventy-fifth and twenty-fifth
22 percentile. And the Puerto Rican box are lower than the
23 other groups. You can see the boxes are all lower.
24 We saw that before, but the boxes themselves are
25 somewhat lower, and Caucasian Americans and Asian Pacific
46
1/17/01 - BENCH TRIAL - VOLUME II
1 Americans have values, and their range of accepted, of
2 accepted applicants is somewhat higher with respect to
3 GPAs. And that's also true of other Hispanic Americans and
4 foreign and unknowns.
5 Q. Now, did you do this similar information, or did you
6 construct similar grids for the later years, 1996 through
7 2000?
8 A. My reports contain plots like this for every year,
9 yes.
10 Q. We don't have all those on your presentation here
11 today, the slide presentation?
12 A. I think it's probably best we don't show all those,
13 but they're all in the reports.
14 Q. Could you summarize in a very conclusionary fashion
15 whether your conclusions were the same, were the same or
16 similar with respect to what you found in these later
17 years, 1996 to 2000?
18 A. There's no substantive difference in the conclusions
19 for each, each of the substantive years.
20 Q. You can find these box plots in your reports?
21 A. Yes. The four for 1995 through '98 are in the first
22 report. And the one for 1999 is in the report referring to
23 1999 analysis. The one for 2000 is in the report referring
24 to the 2000 applicants analysis.
25 Q. Can you just explain why it is that, that your, this
47
1/17/01 - BENCH TRIAL - VOLUME II
1 data is contained in several different reports? Just give
2 is an explanation for that?
3 A. Why it's in several reports?
4 Q. Several different reports for several different years?
5 A. Well, when I constructed the first report, I only had
6 the data for 1995 through '98. Later on I got the data for
7 1999 and conducted a report for that and later I got the
8 data for 2000 and conducted a report for that.
9 Q. Let's go to the next slide. What was the next step in
10 your analysis?
11 THE COURT: Excuse me. Hey, Len, if you guys want
12 to move over to the jury box, too, you're more than welcome
13 to.
14 MR. NIEHOFF: Thank you, sir. I appreciate it.
15 THE COURT: Excuse me. I saw them struggling with
16 the book.
17 MR. KOLBO: We have more copies of this
18 presentation.
19 Q. Can you tell us, Dr. Larntz, what the next step of
20 your analysis was, as reflected by slide fifteen?
21 A. Right. This is a box plot constructing the same way
22 for accepted applicant LSAT scores. And we can see the,
23 the same pattern in the sense that there are four boxes
24 that are lower than the others; those for Native Americans
25 accepted applicants, African-Americans accepted applicants,
48
1/17/01 - BENCH TRIAL - VOLUME II
1 Mexican American accepted applicants and Puerto Rican
2 accepted applicants.
3 And in fact, I guess in this case, the boxes
4 themselves are all at or below the lower end of the boxes
5 for Caucasian Americans and Asian Pacific Island American
6 applicants. That means in terms of what we're talking
7 about, the seventy-fifth percentile, the upper one of the
8 box, the seventy-fifth percentile for these four groups
9 individually are lower than the twenty-fifth percentile of
10 accepted applicants for Caucasian Americans, Asian Pacific
11 Island Americans.
12 Q. And did you construct similar grids and do similar
13 analysis for the later years as well?
14 A. Yes. Lots for 1996, 1997, 1998, 1999 and 2000 are
15 contained in my reports.
16 Q. And the conclusions there, are they the same or
17 similar?
18 A. The substantive conclusions stay the same. Obviously
19 the numbers and the actual plot positions change somewhat,
20 as we'd expect to vary from year to year, but the
21 substantive conclusions remain the same, yes.
22 Q. Okay. Let's go to the next slide. Dr. Larntz, can
23 you tell us whether slide sixteen reflects the next step
24 that you took in your analysis?
25
49
1/17/01 - BENCH TRIAL - VOLUME II
1 (Whereupon an off-the-record
2 discussion was had.)
3 Q. And the question again is if you can tell us what,
4 what your next step in the analysis was and how it's
5 reflected on slide sixteen.
6 A. Well, what I did, this was actually a grid. I call it
7 an admission grid of LSAT and GPAs for all applicants in
8 1995. And this is a, this is actually my construction
9 duplication of a grid that was given to me when I
10 originally got materials for the case of an admission grid
11 that was constructed by, I presume, by the admissions
12 office of a law school.
13 And what the grid gives is for, it cross
14 classifies individuals by LSAT score. And there are a
15 number with no LSAT score in the range that we think of
16 from 120 to 180, and then classifies individuals by LSAT
17 score for categories 120 to 145, out through, well, this
18 doesn't look very good on here, but it's 170 and above.
19 Q. It's the last category.
20 A. So it classifies applicants by their LSAT score in
21 relatively small ranges of LSAT, and their, cross
22 classifies them by grade point average. So the first line
23 for grade point average is 3.75 and above, and then by
24 quarter grade points down until we get to below two. So
25 for instance, and I don't know, I don't know if you can see
50
1/17/01 - BENCH TRIAL - VOLUME II
1 the lower right-hand corner or not.
2 THE COURT: I can see it on my own chart.
3 A. Okay. In the lower-right hand there are all together
4 4,147 applicant.
5 THE COURT: Right.
6 A. And we cross classified the individual applicants, and
7 I also put in this chart the number admitted, and the
8 number right below is the number admitted. So there are
9 1,130 admitted. And for individual cells, we can go at,
10 for instance, at a combination and we highlighted one,
11 individuals with a 3.25 to 3.49 GPAs and LSAT of 161 to
12 163, in that range. There were 198 applicants overall,
13 198, 17 of whom were admitted.
14 So with my charts from before, you could calculate
15 the odds by just looking at, it would be a ratio of 17
16 divided by, I'll do my math, 181, so you could actually
17 calculation the, the observed odds of admission for a case.
18 Q. Can I ask you, Dr. Larntz, how it was you decided to
19 display your materials in this graphic form? I think you
20 may have alluded to it, but could you give a more detailed
21 explanation as to how it is you came to explain the
22 information in this format.
23 A. Thank you. Well, this is exactly the format that I
24 received. I think we, it was referred to me as Exhibit 16
25 from the law school. I'm not sure what exhibit number that
51
1/17/01 - BENCH TRIAL - VOLUME II
1 was or where it came from.
2 Q. Dwayne, could you show the witness Exhibit 16, if he
3 doesn't have that in front of him.
4 A. I have it in front of me here.
5 Q. And do I understand that you did your own work with
6 database to construct a grid in the same fashion as is
7 displayed on Exhibit 16?
8 A. Yes. I used the database, the computerized database
9 that was provided to me and reconstructed Exhibit 16 to my
10 own satisfaction that these numbers were exactly the same.
11 Q. And was it with the use of Exhibit 16 that you decided
12 upon the manner in which the different LSAT combinations
13 and grade point averages should be combined or put together
14 in a cell?
15 A. I used exactly the same categorization as Exhibit 16,
16 yes, I used, that's what I used.
17 Q. If we go to the next slide then, Dwayne.
18 A. Exhibit --
19 Q. Or tell us what slide 17 represents.
20 A. In the, in Exhibit 16 there are further breakups of
21 the all applicants into various sub-groups, and sub-groups
22 by ethnic group and sub-groups by gender and combinations
23 of ethnic group and residency. And so what, and the next
24 page in Exhibit 16 is the admission grid for Native
25 American applicants only. You can see there are, what, in
52
1/17/01 - BENCH TRIAL - VOLUME II
1 this case, 45 applicants, fourteen of whom were admitted
2 overall. And in the cell we were following, there were two
3 applicants, two of whom were admitted. So both were
4 admitted.
5 Q. And how would you summarize the odds for that
6 particular, that particular cell?
7 A. If I were calculating odds, now we're into
8 mathematics. So it's two versus zero, right, two admitted
9 and none denied. And so technically, the odds in that
10 case, if we divide by zero, they turn out to be infinity.
11 Obviously that's, that's a big number, if I want
12 to say that, in statistical terms, but, in fact, we know
13 that with small numbers that we have here, we're going to
14 see numbers like that, numbers like infinity. I would
15 believe that if there were many more applicants, and there
16 weren't. But if there were many more applicants, we would
17 probably see some accepted and some denied. The odds would
18 be different than infinity.
19 Q. The next slide there. What does slide 18 demonstrate?
20 A. This is the, again, in the next, Exhibit 16, chart was
21 for African-Americans applicants and they're in this order
22 because that's the order at which the variable is in the
23 database. And that's the order in which they were
24 displayed in Exhibit 16.
25 And so we have African-Americans applicants.
53
1/17/01 - BENCH TRIAL - VOLUME II
1 There were 404 applicants, 106 of whom were admitted. And
2 in the cell we're following, there were four applicants,
3 four of whom were admitted.
4 Q. Next slide?
5 A. This is the cell for Caucasian American applicants.
6 Again, remember we had 2,316 applicants, 668 of whom were
7 admitted. And in the cell we're following, there were 126
8 applicants, five of whom were admitted.
9 Q. And what did you calculate the odds for that
10 particular?
11 A. If I calculated the odds, it's five divided by 121,
12 five divided by 121. I could do that with a calculator.
13 I'm not going to try to do in my head.
14 THE COURT: You can use it if you wanted to.
15 Q. Why don't you go ahead and tell us what that number
16 represents.
17 THE COURT: He doesn't have to. I just thought if
18 he wanted to.
19 MR. KOLBO: I'd just as soon, Your Honor.
20 A. Five divided by 121 is 0.041 so in odds of 0.041.
21 Q. Okay. Then if we could proceed to the next slide.
22 What does this information tell us?
23 A. The next slide is for Mexican American applicants.
24 There are 98 total applicants, 41 of whom were admitted.
25 And in the cell that we're just going through to summarize,
54
1/17/01 - BENCH TRIAL - VOLUME II
1 there was one applicant and that applicant was admitted.
2 Q. And the next slide?
3 A. Other Hispanic Americans, as classified in the
4 database, 115 applicants, 15 of whom were admitted. And in
5 the cell that we're looking at, seven applicants, one
6 admitted.
7 Q. And the next slide.
8 A. This is Asian Pacific Island Americans, as tabulated
9 in the database. There was 470 applicants, 111 of whom
10 were admitted. And in the, again, the grid cell we're
11 following, there are twenty applicants, two of whom where
12 admitted.
13 Q. And could you calculate the odds on this?
14 A. This one I can actually calculate, because that's,
15 that corresponds exactly to what we had before, isn't that
16 right, a ten percent chance of, ten percent chance, so two
17 to eighteen, so the odds are one ninth, one ninth.
18 Q. And the next slide.
19 A. This would be for Puerto Rican applicants. And there
20 are a few, twenty, not a lot of individuals classifieds as
21 Puerto Rican Americans. Twenty were, there were twenty
22 applicants, five of whom were admitted. And in this
23 particular cell there weren't any. So as far as
24 information about Puerto Rican American applicants, this
25 cell doesn't tell us anything, because there aren't any.
55
1/17/01 - BENCH TRIAL - VOLUME II
1 Q. Okay. And next.
2 A. Foreign applicants, overall twelve applicants, twelve
3 of whom were admitted, and the cell we're following, three
4 applicants, none of whom were admitted.
5 Q. Okay. Let's go to the next cell, or next slide then.
6 A. And finally, this is the last one in this series that
7 give, that give the nine, by ethnic groups. There are 560
8 individuals classifieds as unknown ethnicity, 158 are
9 admitted, and in the cell we're following, 35 applicants,
10 two of whom where admitted.
11 Q. Next, Dwayne. What does this slide summarize?
12 A. This slide is just a summary, looking at that, the
13 particular cell that we happened to follow through in the
14 database. The 3.25, 3.49 GPAs, LSAT range of 161 to 163,
15 and this just summarizes the number of applicants for each
16 of the ethnic groups and the total, and the number of
17 accepted applicants for each of the ethnic groups, among
18 those who applied.
19 Q. Okay. Let's go to slide 27 then. Can you?
20 A. Well, this.
21 Q. Tell us what the next step in your analysis was, Dr.
22 Larntz.
23 A. If we wanted to compare, say, two ethnic groups with
24 respect to their chance of admission with similar
25 credentials; that is, similar GPAs and similar LSAT scores,
56
1/17/01 - BENCH TRIAL - VOLUME II
1 classified in the same way that the law school did in their
2 Exhibit 16, we could, for instance, put the tables side by
3 side, and try to read them, which would be a nice exercise.
4 I don't know how readable they are in the written report.
5 But we could, we could, for instance, look at,
6 say, well, I think we have it set up to bring up the, the,
7 the actual comparison by, say, all the individuals with
8 LSAT scores of 161 to 163 for both groups.
9 So we could, for instance, look at the, at the
10 comparison of African-Americans applicant, and their
11 admission to Caucasian American applicants and their
12 admission. So, for instance, for individuals with that
13 range of LSAT score and grade point averages in the highest
14 category, 3.75 and above, for African-Americans applicants,
15 it was three admitted out of three. And for Caucasian
16 American applicants, it was eight out of 93.
17 In the next cell down, 3.5 to 3.74 grade point
18 average, among African-American applicants that there were
19 six applicants, five of whom were admitted and among
20 Caucasian American applicants, there were 161 applicants,
21 14 of whom where admitted.
22 Going down to the next grade point average for
23 African-Americans, four out of four were admitted. And
24 Caucasian American applicants, five out of 126. That's the
25 cell we were following before, exactly the same, so the
57
1/17/01 - BENCH TRIAL - VOLUME II
1 numbers are the same.
2 To the grade point average range from three to
3 3.24, seven out of eight African-Americans admitted, and
4 two out of 42 caucasian Americans admitted. For 2.75 to
5 2.99, African-Americans, there were four applicants. All
6 four were admitted. Caucasian American applicants, there
7 were 14 in that combination, none of whom were admitted.
8 Grade point average of 2.5 to 2.72,
9 African-Americans admissions were two out of three,
10 Caucasian Americans, zero out of seven.
11 And in the next one, which is 2.25 to 2.49 there
12 was one African-American applicant who wasn't admitted, and
13 five Caucasian American applicants, none of whom were
14 admitted. And so, in fact, in this cell, there's really no
15 ability to compare admission rates in the sense that
16 they're both the same, but there's no, no one was admitted
17 in that particular, no one is admitted in that particular
18 cell.
19 In order to make a comparison, at least in terms
20 of an odds ratio, we have to have individuals that two,
21 each ethnic group in the class, in the cell, and we also
22 have to have some admitted and some denied. And so in this
23 case, both were denied. The next cell has no applicants
24 and no, well, you can see what goes on down below.
25 Q. If you have a, if you have, if you're comparing two
58
1/17/01 - BENCH TRIAL - VOLUME II
1 groups and all are admitted from one cell and all are
2 admitted from that same cell in the other group, can you
3 compute relative odds for that, for those two groups?
4 A. Actually the relative odds is, it could be calculated
5 as being either an infinity, overinfinity in the case
6 you're saying, if all were admitted, and we wouldn't, and
7 we would say that that cell didn't have any comparative
8 information with respect to relative odds.
9 Q. And the same is true if you have cells in which no
10 one's admitted from either group?
11 A. Again, in that case if no one is admitted, the odds
12 are zero for each of the cells. Then it would be a zero
13 over zero. We wouldn't define that either. We would say
14 that also gives no comparative information.
15 Q. Okay. Where there's comparative information, that's
16 when you calculate, when you can calculate relative odds?
17 A. Where there's comparative information, you can
18 calculate relative odds. Although you may wind up as we
19 did in our example with some examples with small counts of
20 infinities or corresponding into zeros.
21 Q. Okay. For example, the first, at the very top there,
22 three over three, and 93 over eight, would that calculate
23 out to infinity because in the case of the African-American
24 applicants one hundred percent of them were admitted?
25 A. That's correct. You'd wind up with three over zero,
59
1/17/01 - BENCH TRIAL - VOLUME II
1 which would be infinity, divided by 91 over 85, which is
2 not, and so we'd wind up with an infinity there, yes.
3 Q. And the cell below that where there are, where it's
4 less than a hundred percent for either category, could you
5 calculate relative odds for the six over five and the 161
6 over 14?
7 A. Yes, you could.
8 Q. Using the same method that you've explained earlier?
9 A. Right. It would be five over one, so there would be,
10 five would be the odds, five would be the odds. They
11 observe odds of African-Americans admits and five over 121
12 would be the odds of admission for Caucasian American
13 applicants.
14 Q. Actually it's the cell above?
15 A. I'm sorry. I'm glad someone's checking me when I read
16 these. I apologize. It's, you're right. It's 14 out of a
17 hundred and, now I lose my train of thought, 149, I
18 believe.
19 Q. All right. Let's go to the next slide then. Can you
20 do the same, you've illustrated these comparisons with
21 respect to a column of LSATs. Can you do the same thing,
22 illustrating a column, a horizontal column with grade point
23 average ranges?
24 A. Exactly. I think we're set to look at those for the
25 same range that we did before.
60
1/17/01 - BENCH TRIAL - VOLUME II
1 Q. And what, what information is summarized here?
2 A. Are we waiting for another one to come up?
3 Q. Sure.
4 A. Thank you. What we've done here is for the grade
5 point average of 3.25, 3.49, we've looked at the, across
6 LSAT score combinations as laid out in the Exhibit 16
7 categories and we've compared African-American admission
8 chances to those of Caucasian applicants with the same
9 grade point average, and then we can see how it changes is
10 a function, or as LSAT changes.
11 So in essence here, if we look across the row for
12 African-Americans applicants, we can see that those
13 African-Americans applicants were very low LSATs
14 relatively. There's 120 to 140. There's, well, none of
15 them were admitted. There were 15 applicants. And
16 similarly in the next category, none were admitted.
17 And as we go across, two out of six in the next
18 cell were admitted, three out of seven in the next one,
19 four out of five. And then actually in this case, once you
20 get to 156 and above, it looks like in this case, all the
21 African-American applicants with LSATs in that range, and
22 that value or above, ten out of ten, three out of three,
23 four out of four, one of one, two out of two, all of the
24 were admitted.
25 So you can see as LSAT increases, the chance of an
61
1/17/01 - BENCH TRIAL - VOLUME II
1 African-American applicant being admitted increases. And
2 that's consistent with, I think, the stated law school
3 policy. And similarly we can look at the same, at the same
4 increase for caucasian applicants.
5 It turns out that, that all Caucasian American
6 applicants below 155 are not admitted except for those for
7 whom there was a no-LSAT category, so at least all of those
8 that had numerical value from 120 to 155, none of those
9 were admitted. And then admissions started in the 156
10 range, exactly the same category in which all
11 African-American applicants then are admitted.
12 And then we had one out of 51 compared to ten out
13 of ten, one out of 61 compared to five out of 126, compared
14 to four out of four, eleven out of 92, compared to one out
15 of one. 38 out of 78, compared to two out of two. And
16 actually the last cell has 55 over 74, but there weren't
17 any African-American applicants in that cell.
18 Q. Let's go to the next slide down. You spent some time
19 taking a look at these cells and some comparisons. If you
20 could just remind us again how relative odds would be
21 calculated looking at those comparisons.
22 A. All right. This should be a summary exactly of the,
23 of what I did on the board earlier; that is, that if we
24 look at calculating relative odds, relative odds comparing
25 a 75 percent probability of admission to a 75 percent
62
1/17/01 - BENCH TRIAL - VOLUME II
1 probability of admission corresponds to a relative odds of
2 nine, just exactly the same way we did before. And
3 similarly, a relative odds of a 90 percent probability of
4 admission, two out of a ten percent probability of
5 admission the relative odds would correspond to 81.
6 Q. You can go to the next slide then, Dwayne. Can you
7 tell us, Dr. Larntz generally what kind of analysis then,
8 perhaps you can draw us some detail about what kind of
9 overall analysis we did to assess the relative odds of
10 acceptance for the different racial groups that you studied
11 or other groups that you studied, as well.
12 A. Well, what we wanted to do next, or what I wanted to
13 do next, and what I did next was to say, try to provide a,
14 what I'll call a composite measure of the relative odds.
15 Many of these cells are very small, relatively small. They
16 have small numbers of counts and small numbers of
17 individuals. Some cases there are cells where there are no
18 admissions. Some cases there are cells where there are a
19 lot of admissions.
20 And so what I did was construct a statistical
21 model that allowed us to calculate a composite relative
22 odds combining all the cells, all the cells in the grids
23 that have comparative information. So in all the cells
24 where there was comparative information, they would
25 contribute to a composite relative odds.
63
1/17/01 - BENCH TRIAL - VOLUME II
1 And what I wanted to do was summarize, I mean, the
2 relative odds that we saw before, some were very, some, a
3 lot were infinity. Some were other numbers. Can we put
4 those together and the statistical method I used which was
5 logistic regression allows us to create an estimate of the
6 relative odds comparing the odds of admission from one
7 ethnic group to another ethnic group.
8 And the goal of that analysis was to summarize
9 the, the weight or the effect in terms of relative odds
10 that are given to members of various ethnic groups with the
11 same grade point average range as we did in Exhibit 16 and
12 the same LSAT range. So individuals that have the same
13 credentials with respect to those two variables.
14 Q. Now, you've mentioned that there are occasions when
15 you get a calculation of infinity because, for example, all
16 applicants from one minority group might be admitted and
17 you have less, you have some number of majority applicants
18 who are denied admission and you'll get an infinity value,
19 correct?
20 A. That's correct.
21 Q. And how is that accounted for in your assessment of
22 trying to summarize in overall fashion the relative odds of
23 different racial groups when you have cells that shows
24 infinity as a value for relative odds?
25 A. Well, those cells contribute information. They're
64
1/17/01 - BENCH TRIAL - VOLUME II
1 not, they're not no-information cells. They certainly
2 contribute information. And so the statistical technique
3 that's used, logistic regression makes those a composite,
4 makes a composite estimate from those cells accounting for
5 all the information, all the cells that have comparative
6 information. And so, for instance, if I might use a very
7 mundane example, and I apologize, Your Honor, if you're not
8 a Chicago Cubs fan, but I go to a Chicago Cubs game every
9 year.
10 I was a graduate of the University of Chicago and
11 they collapsed in 1969 when I was there. And so I followed
12 them from then on. And I have a son who was born in
13 Chicago, doesn't live there any more, lived there only a
14 couple years, but he said that's his hometown, so we have
15 to go back.
16 THE COURT: Oh, yeah. I had a law clerk who
17 wanted to visit every single -- I'm not a baseball fan, and
18 she wanted to visit every single baseball stadium in the
19 country. So we traveled quite a bit hearing cases all
20 over. So I've been to many stadiums. I don't think it was
21 a Cub game. Steve, you want went to a Cub game, didn't
22 you?
23 (Whereupon an off-the-record
24 discussion was had.)
25 THE COURT: But not with Barb. She'd always say I
65
1/17/01 - BENCH TRIAL - VOLUME II
1 need to, assign it here, because I hadn't seen that
2 stadium, and then they started building new stadiums. She
3 just went crazy because, you know, we'd have to go back to
4 cities again. Anyhow, go on. I've been to more games than
5 I wanted to with her.
6 A. Well, I have to do as a father, of course. But, and
7 so I sacrifice myself for that, but I don't -- anyway.
8 At any rate, we've sometimes watched Sandy Sosa
9 play. And we are very curious while he plays each day.
10 And sometimes he does very well. And for instance, he, he
11 bats thousand in some individual games. That's true. I've
12 seen him do that, hit three home runs and bat a thousand.
13 That's a pretty good day for Sandy Sosa. That's a pretty
14 good for most baseball players, right.
15 And so those a thousands are like, are like
16 infinities, in some sense. They're sort of off the scale.
17 And we've also seen Sammy Sosa on other days, I have to
18 admit, strikes out every time at bat, and that's like a
19 zero, nothing, no. And so neither one of those probably
20 represent his batting average overall, right, because, you
21 know, a batting average overall is a composite of a large
22 series of games.
23 And these numbers that I have here, and we
24 certainly would include the games where he, he did very
25 well, and the games where he didn't do very well. And
66
1/17/01 - BENCH TRIAL - VOLUME II
1 these estimated relative odds that we have here are
2 somewhat composites of the relative odds that we see in
3 the, in the individual cells.
4 Q. So in calculating Sammy Sosa's batting average, you
5 don't throw out the games where he bats a thousand, do you?
6 A. No.
7 Q. You don't throw out the games where he bats a zero?
8 A. No, you don't.
9 Q. You include those and they get factored into the
10 analysis?
11 A. That's correct.
12 Q. Is that the same here with respect to the analysis
13 that you did?
14 A. In some sense that's the same. We've got an infinity
15 and a zero, corresponding to the same kind of thing in our
16 case. They correspond to boundary cases, that is, in some
17 cases, a minority sub-cell where we had a hundred percent
18 admitted. That would correspond to an odds ratio of
19 infinity. We certainly include all that information in
20 this composite.
21 Q. Now, you've provided some information about how
22 relative odds are calculated. And we've seen some
23 information about that on the board.
24 Can you tell us how, outside the context of a case
25 like this, how would the other areas where you have
67
1/17/01 - BENCH TRIAL - VOLUME II
1 performed statistical work, whether it's regulatory work or
2 medical work, medical devices, how are relative odds used
3 in those fields, generally speaking?
4 A. Well, actually, in a clinical study, say of a new
5 medical device or a new drug where we would be comparing
6 the new drug or new device to a standard therapy or a, a
7 drug case may be a placebo pill or a current active pill,
8 we would compare the results in, in exactly the same way.
9 We would use our individual cells, in this case
10 would be hospitals, hospitals, for each hospital. If we
11 did a multi-center study of many hospitals, and I've been
12 involved in studies that include, you know, fifty or a
13 hundred hospitals, each hospital contributes a, some number
14 of cases to the study, maybe not very many, just like a
15 cell contributes some number of cases. And we would look
16 at, in each hospital, the number of individuals in the,
17 that took part in the study, that received, say, the
18 standard treatment and seeing what happened to them,
19 whether or not they were cured or had some complication or
20 something like that.
21 And we'd also look at the number with a new
22 device, the number that took part in that particular
23 hospital, and the number that were, say, cured, or had a
24 complication. And many times, many times these studies
25 result in a number of fairly small cells with not very
68
1/17/01 - BENCH TRIAL - VOLUME II
1 large numbers, very similar to what we have here.
2 And we put together, using this, the technique I'm
3 using here is a technique we use to get a composite
4 relative odds of how useful the new device or the new drug
5 is compared to the standard. And we, we often summarize
6 that in terms of relative odds.
7 Q. Can you give some examples of what are, in, say, in
8 the field of working with medical devices or new medicines,
9 what kind of odds are deemed significant? What kind of
10 relative odds are deemed significant in those fields?
11 A. Well, I mean, if I were designing a study, if I were
12 designing a study, we'd often make, you know, some kind of
13 estimate of what was an important relative odds with
14 respect to patients.
15 And depending on the area, relative odds, I'd
16 certainly design a study where we're looking for a relative
17 odds of say, two, or one and a half. Or, I read about Dick
18 Cheney's angioplasty, using a stint. And it's an area I
19 work in, actually, coronary stints, and there the, the
20 relative odds of heart disease versus no heart disease for
21 someone, say, receiving Aspirin is about 1.3, 1.4.
22 So relative odds that might be small numbers
23 greater than one are common. Odds -- I can think of a
24 medical example I had once, if I might continue, where we
25 were looking at historical data of people who were at very
69
1/17/01 - BENCH TRIAL - VOLUME II
1 low blood pressure, very low cholesterol, and their chance
2 of a heart attack, or heart disease, showing itself, to
3 those with high cholesterol and high blood pressure. And I
4 calculated relative odds in a case like that, the actual
5 observed relative odds was, well, if I remember right,
6 about nineteen. We used that as an example in my classes,
7 because that's such a big number.
8 Nineteen's an enormous relative odds, and so that
9 would. To me, it gave me advice on how to handle my diet,
10 of course, and things like that, but it was some of the,
11 some of the odds we're seeing, if you change relative odds
12 by factors of one and a half, two, twenty would be
13 incredibly large value in medical studies I've dealt with.
14 Q. Well, in the relative odds analysis that you performed
15 in this case with respect to the University of Michigan Law
16 School, can you summarize what your conclusions and
17 findings were for comparisons to the different racial
18 groups?
19 A. This slide 30 is a result in my statistical analysis,
20 using logistic regression, using the technique that I would
21 use in standard medical studies to, to compute composite
22 relative odds or odds ratios. And when you're comparing,
23 when you're comparing two probabilities, you have, you
24 compare one probability one odds to another. And you have
25 to have a baseline of comparison. And so I chose as my
70
1/17/01 - BENCH TRIAL - VOLUME II
1 baseline for comparison, it's typical of the analysis we
2 do, we choose, say, the largest, the largest group as a
3 baseline.
4 So what I did is largest group which is Caucasian
5 Americans I use as a baseline, and they would be given a
6 relative odds to themselves of one. That's by definition.
7 So the one that's given here is just our baseline for
8 comparison. And what I've done is computed the composite,
9 estimated, the composite relative odds for Native Americans
10 to Caucasian Americans as about 61. And African-Americans,
11 the estimated relative odds, this is looking at the
12 relative odds of acceptance, a composite measure across all
13 these grid cells. So we're controlling for, we're looking
14 at combining the information from each combination of GPAs
15 and LSAT.
16 So putting those together for African-Americans,
17 the number is 257. For Mexican American, the estimated
18 relative odds is 81. Other Hispanic Americans it's 1.03.
19 One corresponds to basically a relative odds,
20 which is similar probabilities. One is what you get for a
21 relative odds that are, that they're basically the same.
22 1.35 is the number for Asian Pacific Island Americans, 37
23 for Puerto Rican Americans. 0.5, less than one seems to
24 indicate that foreign applicants, although it's not a very
25 big amount less than one, would be, have a smaller chance,
71
1/17/01 - BENCH TRIAL - VOLUME II
1 compared to caucasian Americans. If the relative odds is
2 below one, then it would indicate that there would be a
3 smaller chance compared to Caucasian Americans.
4 And those unknown the estimated relative odds is
5 1.18. So what I see here in this, in this example, 1995, I
6 see that four of these values are very, very large, very
7 large by all my experience. And it just, how do I say
8 this. These are in non-technical terms, enormous values.
9 There's four that are very, very large. Those for Native
10 Americans, African-Americans, Mexican Americans and Puerto
11 Rican Americans.
12 And so given the same credentials, given the same
13 credentials, that is the same GPAs, LSAT cell, it, there is
14 a tremendous advantage, or allowance made. I'm reporting
15 just what the admissions committees have done with respect
16 to admission. There's a tremendous advantage given to
17 Native American, African-American, Mexican American, Puerto
18 Rican American applicants, compared to, well, Caucasian
19 American, other Hispanic, Asian Pacific Island Americans
20 and foreign and unknown ethnicity.
21 Q. Dr. Larntz, can you give us, can you state your
22 opinion as to how the size of these relative odds compares
23 to generally other kinds of odds you've seen in the kinds
24 of work you've done, either as a consultant or as an
25 academic in your thirty years or more as a statistician?
72
1/17/01 - BENCH TRIAL - VOLUME II
1 A. These numbers are just big, I mean they're giant. I
2 can't recall any particular examples of any data set of any
3 size that, where I found relative odds of this magnitude.
4 They're large.
5 Q. Okay. Incidentally, is the data that, size that
6 you're working with here, would you consider it a large
7 sample, small sample, how, for each of these years, and
8 then for the years overall?
9 A. I should let my computer answer that question. We
10 have a lot of data. We have a large number of applicants.
11 The amount of comparative information we have, although the
12 cells themselves are small, the amount of comparative
13 information we have, for instance, for African-American
14 applicants, in particular, is considered a large data cell.
15 Q. Now, you report on slide 30, to the right there, some
16 information on standard deviations. Can you tell us what
17 the significance of those values are here?
18 A. Well, as I said, I tried to explain earlier, we want
19 to see, there's always a chance of variation in the world.
20 That's life. I said it earlier. I'd be out of business as
21 a statistician if there weren't chance of variation, if
22 everybody responded exactly the same. So I like chance
23 variation, okay. It gives me a livelihood.
24 And now what we want to do is say, these relative
25 odds that we see here, if we redid them for a different
73
1/17/01 - BENCH TRIAL - VOLUME II
1 year, or we redid them with the new group, they would
2 change. There's no question about that. We'd get
3 different numbers. That's -- there's variation. And we
4 wouldn't expect them to be the same. And the question is,
5 is there evidence that these are bigger than the baseline
6 of one. Are they really bigger.
7 And so we can do a test to compare what's the,
8 what's the likelihood, what's the chance that we would get
9 odds this big if, in fact, chance alone were operating, if
10 chance alone were operating. So could we get numbers like
11 this. Sure, I mean, you can always get any number. Now,
12 how likely is it that chance alone is operating.
13 So, what I did in the standard deviation column is
14 summarize the degree of statistical evidence in terms of
15 standard deviations, so we're thinking, something
16 statistically significant, if it's two or three, or larger
17 in magnitude, and so what I did is summarize the degree of
18 statistical evidence that, concerning each of the relative
19 odds and how likely it is that chance alone was creating
20 this.
21 Q. And what were your conclusions?
22 A. Well, it's clear there are, in this particular slide,
23 there are four values that are bigger than the usual two or
24 three, and we're talking about. There are four, and those
25 are for Native American, African-American, Mexican American
74
1/17/01 - BENCH TRIAL - VOLUME II
1 and Puerto Rican applicants. And so that's, in fact, the
2 smallest one of those is 4.95, the largest is fourteen.
3 Those are, again, sort of strong, strong indications that
4 it's not just chance alone that's, that's causing these,
5 this difference in relative odds.
6 Q. Now, again, did you perform the same kinds of analysis
7 here of relative odds for acceptance controlling for GPAs
8 and LSAT grid cell for the later years, 1996 to 2000?
9 A. Yes.
10 Q. And we don't have those here.
11 A. I think we do actually have the '96, '97, '98.
12 THE COURT: Let me just ask you a question. In
13 terms of relative odds, can that be translated into how
14 many times more likely it is that one group will be
15 admitted compared to another group?
16 A. It's directly in terms of odds, Your Honor. So if, in
17 fact, so, if -- let's just take the example that we had.
18 So if, say --
19 THE COURT: Using the figures that are on the
20 board.
21 A. Okay. Okay. Sure. Can I use the Mexican American
22 figure?
23 THE COURT: Whichever one you want, just.
24 A. Well, that number there, you see what number that is?
25 THE COURT: Sure.
75
1/17/01 - BENCH TRIAL - VOLUME II
1 A. It's 81. And we've worked with 81 already. So I can,
2 I can do it in that terms, and I think it makes it easier
3 for me to explain. So, for instance, if, if we, if a
4 Caucasian American applicant in a particular combination of
5 GPAs and LSA had a ten percent chance, ten percent