Blog

Guest Post: Be True to Your School: But True to Your Law School?

Guest Post by Stephen Daniels

Senior Research Professor
American Bar Foundation

Be True to Your School was a popular song of an earlier time extolling staunch loyalty to one’s alma mater. But staunch loyalty to your law school these days in light of high tuition, high student debt, concerns about inadequate preparation, and mediocre job prospects? Two recent reports drawing from retrospective surveys of graduates done by Gallup – one as a part of the Gallup-Purdue Index and the other done in conjunction with AccessLex Institute – suggest that many see little value in their legal education given the cost. Those choosing to attend law school make a substantial investment with the expectation of a substantial return. Is there value in the proposition? Is there cause to be true to your school?

I do research on legal education from an institutional perspective and the question about value animates much of my current work involving law students.[1] Students – or more precisely, their tuition dollars – are indispensible for legal education. The LSSSE survey data allow me to explore students’ views over time in search of an answer for that question.

The LSSSE data are important because they provide not just information about students, but information from students. Simply put, the surveys give voice. The data are especially unique because they are longitudinal, the surveys asking students a host of questions over time using the same wording – the gold standard for examining change, or the lack thereof, in views, attitudes, or assessments.

Although the data do not cover all schools, and LSSSE does not claim to have a representative sample of all schools or students, the range of participating schools is wide. The number of respondents in any given year or set of years is large. The 2016 survey, for example, had over 18,000 respondents from 72 schools; in 2010 the figures were just under 23,500 from 73 schools.

My current project using the LSSSE data – Be True to Your School: But True to Your Law School? – explores patterns and changes in law student assessments of their schools. The question about being true to one’s school provides a framework, a heuristic for exploring a range of factors important to students given the investment they are making. In that old song, what made you true to your school was its winning football team. What makes a law student true to their school is the school’s success in meeting their expectations – enough, to paraphrase a LSSSE survey question, that the student would attend that school again if given the choice.

The data I utilize cover the years 2005 through 2016. They include respondents from 38 schools participating in the LSSSE surveys at least three times between 2005 and 2010, and at least three times between 2011 and 2016 (allowing for comparisons before and after the 2010 enrollment peak and subsequent decline). Focusing on those 38 schools—which were not identified, just as the students themselves remain anonymous—provides a reasonable degree of continuity. Not all respondents from those schools are included, just those at the end of their law school experience (usually 3Ls and occasional part-time students who take a fourth year (4Ls) to complete their studies). Structured this way the data set consists of 33,661 survey respondents.

Are 3/4Ls at least reasonably true to their schools? In apparent contrast to the Gallup surveys, preliminary analyses suggest yes – at least in general. Most respondents said they (probably or definitely) would attend the same school again.

Affirmative Answers to Select LSSSE Questions by Time Period

(3/4L respondents in 38 schools)

Additionally, the percentages reflect some stability over time with regard to attending the same school again. The 2011-13 dip reflects the cohort graduating as the post-2010 enrollment declines began, with the next cohort having a somewhat more positive view. It was the effect of a drop in affirmative responses from private school students (from 75% in 2008-10, to 69% in 2011-13). There was no similar decline for public schools (81% in 2011-13, 85% in 2014-16 to 80%). Not surprisingly, the most important factor for an affirmative answer about attending the same school again was a student’s evaluation of their overall law school experience.[2]

While attending the same school is a summary question close to the Gallup surveys’ interest in value, other LSSSE questions probe the idea of value more deeply and are more revealing of the complexities of students’ views. One question asks the extent to which the law school contributed to acquiring work-relevant knowledge and skills. The graph shows that a majority responded affirmatively about their school’s contribution, with an upward trend except for another 2011-13 dip (again, private schools).

This particular contribution was especially important for those respondents saying they would attend their school again, as 70% also answered affirmatively about their school’s contribution to work-relevant knowledge and skills. For those not so interested in attending their school again, only 37% thought their school made that contribution.

Another two questions ask about crucial employment-related matters. One question asked about satisfaction with the school’s help in finding a job for those using it (86% did use that help). As that column shows, most respondents were not enthusiastic in their assessments (with the 2011-13 dip). Students who would not attend their school again were even less enthusiastic about the help provided -- the percentage of them offering a positive assessment ranged only from 20% in 2005-07 to 25% in 2008-10.

The other employment-related question asked about the degree of a school’s support for the student’s own employment search. Most respondents did not find their schools particularly helpful in this regard (with the 2011-13 dip). Those who would not attend their school again were even more negative in their assessments on support. Their assessments ranged from only 11% in 2005-07 to 18% in 2014-16. Even a majority (57%) of those who would attend their school again had a negative assessment of their school on this matter.[3]

The responses to these three questions reflect longer-term areas for concern and may provide a hint as to why respondents in the Gallup retrospective surveys were doubtful about the value of their investment. Again, these are preliminary findings from a project very much in progress, with much left to do. What is clear is the importance of the LSSSE data for digging systematically into the details of what students say about the value of their investment.

[1] Funded in part by a grant from AccessLex Institute.

[2] Spearman’s rho =.68, sig.=.000.

[3] Responses to the employment questions are strongly related, Spearman’s rho = .74, sig. = .000.

by Guest

Guest Post: LSSSE as a Key Tool to Support the Learning Outcomes Enterprise

Guest Post by Jerry Organ

Professor of Law and Co-Director of the Holloran Center for Ethical Leadership in the Professions
University of St. Thomas School of Law (Minnesota)

In 2014, the ABA Section of Legal Education and Admissions to the Bar mandated that law schools adopt learning outcomes.  As of March 1, 2019, over 180 law schools have published learning outcomes on their websites.  The Learning Outcomes Database on the website of the Holloran Center for Ethical Leadership in the Professions -- https://www.stthomas.edu/hollorancenter/resourcesforlegaleducators/learningoutcomesdatabase/  --provides a searchable set of databases of publicly available law school learning outcomes organized under three categories -- learning outcomes associated with

• Standard 302(a) (knowledge and understanding of substantive and procedural law),
• Standard 302 (b) and (d) (legal analysis and reasoning, legal research, problem-solving, and written and oral communication in the legal context (b) along with other related professional skills (d)), and
• Standard 302 (c) and (d) (exercise of proper professional and ethical responsibilities to clients and the legal system (c) along with other related professional skills (d)).

The learning outcomes enterprise contemplates that each law school will engage in an assessment process in which it analyzes the extent to which its program of legal education is helping its law students progress on the law school’s identified learning outcomes.  Having engaged in that assessment effort, the law school then should “close the loop” and figure out whether it needs to make changes in its program of legal education to improve the extent to which its students are making progress on the law school’s identified learning outcomes.   This is an iterative process that should result in continual improvement over time in the extent to which the program of legal education is helping students progress on the identified learning outcomes.

Law schools may feel that there are some traditional measures that can be used to assess progress on learning outcomes associated with Standards 302(a) and 302(b), such as performance in doctrinal courses and legal research and writing, and possibly performance in experiential courses and clinics, and performance on the bar exam.  A significant number of law schools, however, have identified robust professional formation learning outcomes associated with Standard 302(c) and 302(d) for which these traditional measures may not be very helpful, including self-directedness, cultural competence, commitment to pro bono, teamwork/collaboration, and integrity among others.

LSSSE can be an important tool to support the assessment efforts associated with the learning outcomes enterprise, particularly with respect to some of these “professional formation” learning outcomes.

First, within the LSSSE survey instrument itself, there are a number of questions that relate to some of the learning outcomes identified above.

With respect to cultural competence, for example, the following questions could be informative:

• How often have you had serious conversations with students of a different race or ethnicity than your own?
• How often have you had serious conversations with students who are very different from you in terms of their religious beliefs, political opinions, or personal values?
• How often have you included diverse perspectives (different races, religions, sexual orientations, genders, political beliefs, etc.) in class discussions or writing assignments?
• To what extend does your law school emphasize encouraging contact among students from different economic, social, sexual orientation, and racial or ethnic backgrounds?
• To what extent has your experience contributed to your understanding people of other racial and ethnic backgrounds?

While these questions do not speak directly to a student’s level of cultural competence, they can inform whether various interventions or curricular or co-curricular innovations a school has implemented have resulted in more interactions among students of diverse backgrounds and can measure the perception of the school’s commitment to emphasizing interactions among students of diverse backgrounds.

With respect to commitment to pro bono, there are four questions that could be used to assess the extent to which engagements or interventions around pro bono have had a meaningful impact in the student experience:

• How often have you participated in a clinical or pro bono project as part of a course or for academic credit?
• Have you done pro bono work or public service?
• How many hours do you spend in a week on legal pro bono work not required for a class or clinical course?
• To what extent has your experience contributed to your contributing to the welfare of your community?

With respect to teamwork/collaboration, there are a handful of questions that could be instructive:

• How often have you worked with other students on projects during class?
• How often have you worked with classmates outside of class to prepare class assignments?
• To what extent has your experience contributed to your working effectively with others?

Similarly, the following questions may be instructive with respect to integrity:

• To what extent does your law school emphasize encouraging the ethical practice of the law?
• To what extent has your experience contributed to your developing a personal code of values and ethics?
• To what extent has your experience contributed to your understanding yourself?

As noted above in the discussion regarding cultural competence, many of these questions are not direct measures of one’s attitudes or capabilities. Rather they serve as indirect measures of how the students experience the law school culture and the extent to which that culture provides certain opportunities or experiences for students.  For law schools trying to determine whether a given change in course requirements or instructional methodologies or co-curricular programming has had a meaningful impact, these indirect measures can be a helpful supplement to other measures, such as observations of students or reflective writing.

Second, LSSSE supports supplemental question sets or modules.  This use of modules might be an even more powerful mechanism for supporting the assessment process regarding some of these professional-formation learning outcomes.  The module concept generally contemplates that multiple law schools would ask a supplemental set of questions usually focused on a common theme or attribute.  Thus, several law schools that each had a learning outcome associated with commitment to pro bono or teamwork/collaboration or self-directedness could come up with a bank of additional questions focused more significantly on one of those learning outcomes.  This “module” could then be used at each of those law schools to get feedback that is law school specific and comparative.

The most fruitful mechanism for using either existing questions or modules is to participate in multiple iterations of the LSSSE, such as bi-annually, which allows a school to see comparisons over time on two platforms.  First, one can assess the change between first-year and third-year, for example.  Second, one can assess change among first-years and third-years over time.   This can be an especially effective way to assess whether a change in curricular offerings or co-curricular programming has had a meaningful impact.

The simple point I want to communicate is that law schools need to be thinking about LSSSE as a real partner in the learning outcomes enterprise.  In addition, law schools should be thinking about working in partnership with other law schools and LSSSE on modules that could make LSSSE an even more powerful assessment tool.

by Guest

Guest Post: Antecedent Experiences Affecting Belonging in Law School

Guest Post By Elizabeth Bodamer, J.D.

Director of Student Affairs, Indiana University Maurer School of Law
PhD Candidate, Indiana University Bloomington Department of Sociology

As a graduate student, I am very fortunate to have the opportunity to work with LSSSE and other collaborators to explore in-depth the student experience in American law schools today.  Last month, Professor Quintanilla shared the LSSSE data finding that the quality of relationships with faculty, staff, and students predicts students’ sense of belonging. Moreover, the data showed that belonging predicts students’ academic performance. Building upon this research, my dissertation will look at the antecedent experiences that affect belonging in law school.

In my dissertation, I argue that minoritized students must perform a balancing act as they navigate their way through school environments and negotiate their sense of belonging in law school.  I suspect that the balancing act will be seen when the effects of negative experiences and perceptions are counteracted by students' support systems. As students are pulled negatively one way, their support system pushes them back up on the law school balance beam. They teeter between their negative experiences and perceptions and their support systems.

Preliminary findings illustrate that student experiences in law school vary by race/ethnicity, gender, and sexual orientation. For example, students from 20 law schools were asked to indicate how strongly they agree or disagree with the following statement:  “I have experienced bias, discrimination, or unfair treatment at my law school based on my race/ethnicity, gender, gender identity, and/or sexual orientation.” 19.42% of students who identified as gay, lesbian, bisexual, another, and/or questioning and 18.11 % of nonwhite students reported having experienced bias, discrimination, or unfair treatment at their law school. This is almost double of what white students (10.49%) and male students (10.56%) reported.

A similar pattern is seen when students were asked to indicate how strongly they agree or disagree with the following statement:  “I experienced not being taken seriously in a class because of my race/ethnicity, gender, gender identity, and/or sexual orientation.” 22.54% of nonwhite students, 19.73% of women, and 20.25% of students who identified as gay, lesbian, bisexual, another, and/or questioning reported not being taken in class. 9.89% of male students and 12.36% of white students reported having not been taken seriously in class. Compared to white and male students, significantly more minoritized law students reported not being taken seriously in a class because of their race/ethnicity, gender, gender identity, and/or sexual orientation.

Why does this matter? Higher education literature shows that peer support, social and academic interaction, perceived racial tension, relationships, friendships, a solid support system, and climate affect students’ sense of belonging (Green et al. 2018; Murphy and Zirkel 2015; Freeman, Anderman, and Jensen 2007; Pittman and Richmond 2008; Hausmann et al. 2007; Hurtado and Carter 2007; Cabrera et al. 1999). It is important not to simply understand that experiences vary based on race/ethnicity, gender, and sexual orientation, but to explore if and how these experiences have a direct effect on belonging, and consequentially on students’ performance. Examining the student experience is crucial because as Duncan Kennedy said, “One cannot grasp the political significance of legal education without understanding that the future is present within every moment of a student’s experience.”

I look forward to continuing my work with the LSSSE dataset as I analyze how negative experiences and perceptions, such as discrimination experiences, perception of how students are perceived, and perception of school’s commitment to diversity, and support systems directly affect belonging in legal education.

Happy 15-year anniversary, LSSSE!

* Dissertation Committee: Dr. Ethan Michelson, chair (Indiana University Bloomington), Dr. Jennifer Lee (Indiana University Bloomington), Dr. Sylvia Martinez (Indiana University Bloomington), and Victor Quintanilla (Indiana University Maurer School of Law).

**One of the many challenges graduate students face in doing research is finding the data necessary to answer their research questions. However, collaborating with LSSSE has allowed me to collect the necessary data to explore the overarching question, what affects law students’ sense of belonging. For this, I would like to thank the LSSSE dream team, Chad, Jakki, and Meera, for working with me to make this dissertation data possible.

by Guest