Guest Post: Rising With the Tide: Using National and Off-Cycle LSSSE Data to Improve Practice and Scholarship
Rising With the Tide: Using National and Off-Cycle LSSSE Data to Improve Practice and Scholarship
Trent Kennedy, MA, JD
National LSSSE data are a valuable and oft-overlooked resource to improve scholarship and data-informed practice. Their free online Public Reporting Tool gives everyone access to aggregated data from the last eight years of LSSSE administrations, allowing for quick checks and relatively detailed dives. Researchers and administrators who are interested in testing assumptions, building theories across data sources, and leveraging national norms should keep it in mind as their projects take shape.
The demographics of LSSSE participants reflect those of US law students overall (at least as of 2017). A total of 181 US law schools have participated in LSSSE since 2004, and each year’s cohort contains law schools from up and down the US News and World Report ratings (and one of my favorite data visualizations in legal education reminds us that it is a rating), capturing institutional diversity as well as student demographics.
Student Affairs professionals, faculty members, and administrators can use the Public Reporting Tool for a number of different useful functions, which are especially powerful when combined with school-specific data.
Testing Assumptions
National LSSSE data allow users to test myths, assumptions, and meso-ideas about law student characteristics and experiences. For example, after noting that 27.1-29.2% of LSSSE participants between 2014 and 2019 reported being first-generation college graduates, Georgetown Law began to investigate the experiences of our own first-generation students, law student affairs professionals from across the country engaged more deeply with the NASPA Center for First-Generation Student Success, and the AALS Section on Student Services is exploring a collaborative program about the needs of first-generation students writ large. As the old assumptions about our student populations become less and less useful, local and national data will help identify needs and marshal resources for the future of legal education.
National LSSSE data also show a consistent decline in overall satisfaction and preference for the same law school as JD students progress from 1L to 3L year. Students in their 3L year will certainly have more experience with and information about their law school, but local data are needed to know if they will really be the best boosters to alumni or prospective students.
Norming
The converse of assumption testing is establishing norms that can be leveraged to influence student behavior. Social norming is a common technique in public health communication, often used to combat perceived isolation as a barrier to healthier behavior. In 2017, my office was struck by the national data around the LSSSE core survey question 6c “In your experience at your law school, how satisfied are you with: Personal counseling?”. Of 20,574 students surveyed that year, 9,132 indicated that they had not used personal counseling while in law school. The actual satisfaction of the remaining 11,442 students ran the gamut, but careful observers will note that 55.6% of JD students participating in LSSSE indicated some uptake of personal counseling services (as distinguished from academic advising, career counseling, and financial aid advising). With so many of our students convinced that they are the only one struggling, seeing flyers around campus with a majority message to the contrary is a powerful sign that vulnerability and help-seeking are nothing to be ashamed of.
In the same vein, national LSSSE data on law student time use helped us build a norming campaign intended to reduce students’ overall stress and sense of time famine (the belief that you have too much to do and too little time to do it all). A quick run through the 2017 LSSSE data showed us that 1Ls spend an average of 31.93 hours per week reading and preparing for class. With 15 hours of weekly classroom instruction for Georgetown Law 1Ls at the time of the project and even factoring in a full night’s sleep each night, that leaves students with plenty of hours per week of waking time not spent in class or studying. That should be around 65 hours to enjoy their meals, visit the gym, talk with their families, or explore the community around them. From our conversations with students and other LSSSE data, we know they don't feel like they have that much time and spend their hours according to that feeling. We wanted to break them out of the cycle of stress and time famine. We presented the LSSSE data on time spent studying and the ensuing arithmetic on other available time during 1L Orientation in the hopes of relieving perceived time pressure, encouraging self-sustaining behavior, and restoring a sense of control to our 1Ls’ lives. A minority of the student feedback still showed the perennial hobgoblin of anxious peer comparison, but the response from the majority of students (and a few visiting alumni) was positive. They wanted the intended outcome and trusted a norm articulated with national data.
Scholarly Corroboration, Synthesis, and Theory-Building Across Data Sets
National LSSSE data can also be used to corroborate or support local data, allowing for the development of theories across multiple data sets. Law student stress has been a perennial topic of interest, rising in perceived importance as post-recession career pressure continues to haunt current students. Segerstrom’s idiosyncratic instrument revealed time pressure, difficulty of material, and lack of feedback to be chief concerns of law students in the late 1990s but “academic environment” was not. National 2015 LSSSE data from a special module on law student stress reported both the overall level of stress and the major sources of that stress, finding that academic performance and academic workload were the most common concerns while classroom environment/teaching methods was the least common. Flynn, Li, and Sánchez’s 2019 study of law student stress found that workload pressure was the strongest predictor of law student distress, but that course design and grading failed to show statistically significant relationships with the expected distress outcomes. Interested scholars might struggle to meaningfully synthesize Segerstrom and Flynn et al. without the intervening LSSSE data that facilitate a clearer focus on student perceptions rather than the institutional structures that produce them.
Even without timely local data, national and off-cycle LSSSE data enhance other avenues for scholarship and practice. By making the most of that data, legal educators can check their assumptions about student demographics and behavior, make new norms visible to change student attitudes, and connect other data sets into a more coherent (and actionable) landscape. Scholars and practitioners would do well to consider national LSSSE data when looking for insights on a proposal, program, or institutional problem.
Guest Post: What LSSSE Data Can Teach Us About Developing Our Law Students for Influence and Impact as Leaders
Leah Teague
Chair, AALS Section for Leadership
Associate Dean and Professor, Baylor Law School
Leah_Teague@baylor.edu
https://traininglawyersasleaders.org/
In legal education we brag about the leadership positions our graduates hold. We tell our current students they too will be leaders someday. We expect that of them. But what are law schools doing to prepare students for those roles? You might be surprised to know that we do very little in law school to intentionally develop students’ leadership acumen and skills.
One might argue that all of legal education is training for leadership. Lawyers routinely use skills developed in law schools (such as critical thinking, legal analysis and advocacy) to help others. Attorneys also serve others by strategizing, persuading, and ultimately commanding the room, whether it be the boardroom, courtroom, or public arena. These are all foundational leadership skills. So why aren’t we being more intentional about preparing our students for the leadership roles we know they will someday assume?
Perhaps more importantly, why aren’t we teaching our students about the critical leadership roles played by lawyers throughout history and the urgent need for us to continue to do so? With precious little time spent in law school on the role of lawyers in society and increasing emphasis on law as a business and lawyers as legal technicians and specialists, fewer students leave law school with an understanding of our role as protectors of the rule of law, seekers of justice, wise counselors to individuals and businesses, and effective community leaders. While understandable, considering the economic and other competing pressures on lawyers’ time today, but how ironic – and concerning – considering that the majority of law school applicants still state in their personal statements their desire to go to law school to do good – to right wrongs, stand up for those without voice or power, or to make a difference in society.
Lawyers are small in numbers. We are less than one-half of one percent of the population in America. Yet, we are still mighty in impact and influence. But that might not continue. Some are concerned that our influence is already waning. For example, in the mid-19th century, almost 80% of members of U.S. Congress were lawyers. By the 1960s this dropped to under 60%. Currently, it is about 40%. Does having fewer leaders trained and experienced in strategic planning, advocacy, negotiation and civil discourse make a difference? Don’t we need more decision-makers at all levels who are trained in the law and who understand our obligation to society as guardians of democracy?
The question for us as legal educators is how do we better prepare our students for these important roles? To address this important issue, a new Section for Leadership was approved by the American Association of Law Schools. Our section reached out to Meera Deo and Chad Christensen at LSSSE to look at what we can learn from data already being collected by LSSSE. Chad shared his thoughts and findings during a webinar on November 5, 2019.
Chad began with an overview of work being done at the Holloran Center and others to look at professional competencies touted by a series of reports (such as MacCrate and Carnegie), desired by legal employers, and now mandated by ABA Standards 302(c) and (d). Many of these professional competencies are attributes of leadership and are included in the annual LSSSE instrument, including teamwork and collaboration, professionalism and integrity, self-directedness and cultural competencies. Chad shared the specific LSSSE questions related to these attributes and discussed the findings related to those questions from 2009 to 2019. The good news is that Chad’s charts showed improvement over time in almost all of those questions. (You can view the specifics of the LSSSE questions and the results related to these leadership attributes by downloading the Webinar Slides.)
Even more exciting was the example he shared of LSSSE in action. In 2008, Indiana University (IU) 1Ls reported a significantly lower level of teamwork/collaboration than 1Ls at its peer schools. After introducing a first-year, legal professions course in 2009 that emphasized a goal-oriented, collaborative approach to legal work, IU experienced a significant improvement in that category.
We believe the work at a growing number of law schools to introduce leadership development into their programming and curriculum not only will benefit students but also our profession and society. With that said, we are still figuring out how to be most effective and where to focus our efforts. We agree that the goal is to teach our students about the critical role lawyers play in society and our obligation as lawyers to serve, and then to better prepare them for the many opportunities they will have to serve. We want to help our students develop skills and traits that not only will enable them to be more effective in their professional roles but also will better equip them to make a meaningful difference in the lives of the individuals and organizations they serve. We also know that when they do so they will experience the kind of meaningful fulfillment that leads to satisfaction, contentment and happiness. Don’t we all want that for our students? For ourselves?
Our thanks to our co-sponsors for the webinar, LSSSE, AALS Section for Empirical Studies of Legal Education and the Legal Profession, and AALS Section on Law and the Social Sciences. We were excited to find more kindred spirits in our work! Our special thanks to Chad for his personal and vested interest in leadership development for lawyers. We look forward to further collaborations!
We invite all who are interested in introducing or expanding leadership development for law students at your school to join us. Feel free to contact me for more information at Leah_Teague@baylor.edu.
Guest Post: Getting to Excellent
Evan Parker, PhD
Founder
Parker Analytics, LLC
Law school administrators operate in an environment that Dean Camille A. Nelson calls “relentlessly competitive” (See Nelson, “Towards Data-Driven Deaning,” LSSSE, August 22, 2019.) With students taking on sizable debt and an increasingly uncertain legal job market, it’s more important than ever that school administrators and faculty find ways to maximize the value of the law school experience.
What should they do to maximize the student experience? Conveniently, we can ask the students. More precisely, we can study students’ views of their law school experience as captured by the Law School Survey of Student Engagement (LSSSE). One question on the survey asks, “How would you evaluate your entire educational experience at your law school?” LSSSE participants respond with one of four options: Excellent, Good, Fair, or Poor. Using a statistical model, we can identify what factors most strongly differentiate students who say their experience in law school was “Excellent.”
As a rich professional survey with extensive coverage of student issues, LSSSE lets us examine a number of possible drivers of student satisfaction with law school, and highlight those factors that seem to matter the most. I view this as evidence into what students want—and thus, what administrators looking to maximize value should work hard to deliver.
In the analysis to follow, the factors that most strongly predict an “Excellent” response—the top “value differentiators”—are within reach of every law school administrator, albeit addressing these factors requires hard work. In broad terms, this student-driven data indicates that schools must do three things well:
- Build and sustain relationships among students, faculty and professional staff;
- Provide first-rate advising, career counseling and personal support; and
- Develop a broad curriculum that instills work-related knowledge.
Analyzing the Law School Experience
In this analysis, we’ll consider pooled LSSSE data over the 2009 to 2019 period. Figure 1 reports the four responses to the “entire experience” question, illustrating the distribution in percentage terms. Number of students per response is listed in parentheses.
Over the past decade, about one-third of LSSSE participants said that their entire law school experience was “Excellent” (35%). A near-majority of students viewed their law school experience as “Good” (48%), and the remainder were less positive (Fair = 14%, Poor = 3%).
We can use a statistical model to identify factors that are most strongly associated with an excellent law school experience. With more than 60,000 complete survey responses (i.e., 60,000 surveys have complete data), it’s straightforward to perform an exploratory analysis—one that includes all possible inputs into the model using the standard battery of LSSSE questions, 2009 to 2019—82 questions in total.[1] In a sample this large, the fact the model returns information about “statistical significance” is not so meaningful, since most of the factors will reach a level of significance. What is meaningful in the model, however, is its ability to produce “appropriate comparisons” between types of students—comparisons which isolate the importance of specific LSSSE factors by revealing the expected differences in students’ perceived law school experiences. For the purpose of this post, we’ll compare student responses that are typical (at the mean) versus responses that are very favorable (+2 standard deviations above the mean), and all else equal.
Figure 2 reports the top 10 positive predictors of the law school experience response. Reported values are the model-based differences between a typical baseline response (mean) and a very favorable (mean +2 s.d.) Scanning across the items, the three general themes involve (1) interpersonal relationships, (2) perceptions of support, and (3) features of the law school curriculum.
Quality of Relationships. The strongest positive differentiator is question (9a), the quality of relationships with other students. On this item, LSSSE asks students whether their relationships with other students are unfriendly or friendly using a 1 to 7 scale. So, at schools where students report above average friendliness among their peers, they are also more likely to view their law school experience as excellent. The same is true for relationships with faculty (9b) and administrative staff (9c), who are evaluated in terms of their availability and helpfulness, respectively.
Counseling and Support. A second set of factors that are strongly positively related pertain to students’ perceptions of support. This is the case in general per the importance of (8c), about the school’s ability to provide necessary support to succeed (1 = Very Little, 4 = Very Much). It’s also the case with academic advising (6a), personal counseling (6c), and career counseling (6b). Perhaps not surprisingly, the quality of a school’s advising and support functions matters a great deal to students.
Law School Curriculum. A final set of factors pertain to the nature of the curriculum. Students who most strongly believed they had acquired a broad legal education (10a) along with job or work-related knowledge (10b), and were taught to think critically and analytically (10b), were those who were most likely to give their law school an overall rating of “Excellent.”
What Students Want
With this short analysis, we see the value in leveraging the student perspective to focus the efforts of law school decision-makers. Students want to attend a school where relationships with other students, faculty and staff are both possible and enjoyable. For Deans, this might mean giving more weight to social functions and community-building activities than they have in the past. Students also want a sense their school is supportive, and working toward their success by providing resources to aid in both academic and personal decisions. They find greatest value when the law school curriculum equips them with skills they will need in practice. At the end of the day, “the ask” seems reasonable on its face.
___
[1] The model is a linear mixed effects regression that accounts for law school class year, survey year, and unique law school effects. More sophisticated models are certainly possible (e.g., an ordinal response model), yet in my exploration I’ve found that the conclusions shared here end up the same regardless.