Guest Post by Jerry Organ

Professor of Law and Co-Director of the Holloran Center for Ethical Leadership in the Professions
University of St. Thomas School of Law (Minnesota)

In 2014, the ABA Section of Legal Education and Admissions to the Bar mandated that law schools adopt learning outcomes.  As of March 1, 2019, over 180 law schools have published learning outcomes on their websites.  The Learning Outcomes Database on the website of the Holloran Center for Ethical Leadership in the Professions — https://www.stthomas.edu/hollorancenter/resourcesforlegaleducators/learningoutcomesdatabase/  –provides a searchable set of databases of publicly available law school learning outcomes organized under three categories — learning outcomes associated with

  • Standard 302(a) (knowledge and understanding of substantive and procedural law),
  • Standard 302 (b) and (d) (legal analysis and reasoning, legal research, problem-solving, and written and oral communication in the legal context (b) along with other related professional skills (d)), and
  • Standard 302 (c) and (d) (exercise of proper professional and ethical responsibilities to clients and the legal system (c) along with other related professional skills (d)).

The learning outcomes enterprise contemplates that each law school will engage in an assessment process in which it analyzes the extent to which its program of legal education is helping its law students progress on the law school’s identified learning outcomes.  Having engaged in that assessment effort, the law school then should “close the loop” and figure out whether it needs to make changes in its program of legal education to improve the extent to which its students are making progress on the law school’s identified learning outcomes.   This is an iterative process that should result in continual improvement over time in the extent to which the program of legal education is helping students progress on the identified learning outcomes.

Law schools may feel that there are some traditional measures that can be used to assess progress on learning outcomes associated with Standards 302(a) and 302(b), such as performance in doctrinal courses and legal research and writing, and possibly performance in experiential courses and clinics, and performance on the bar exam.  A significant number of law schools, however, have identified robust professional formation learning outcomes associated with Standard 302(c) and 302(d) for which these traditional measures may not be very helpful, including self-directedness, cultural competence, commitment to pro bono, teamwork/collaboration, and integrity among others.

LSSSE can be an important tool to support the assessment efforts associated with the learning outcomes enterprise, particularly with respect to some of these “professional formation” learning outcomes.

First, within the LSSSE survey instrument itself, there are a number of questions that relate to some of the learning outcomes identified above.

With respect to cultural competence, for example, the following questions could be informative:

  • How often have you had serious conversations with students of a different race or ethnicity than your own?
  • How often have you had serious conversations with students who are very different from you in terms of their religious beliefs, political opinions, or personal values?
  • How often have you included diverse perspectives (different races, religions, sexual orientations, genders, political beliefs, etc.) in class discussions or writing assignments?
  • To what extend does your law school emphasize encouraging contact among students from different economic, social, sexual orientation, and racial or ethnic backgrounds?
  • To what extent has your experience contributed to your understanding people of other racial and ethnic backgrounds?

While these questions do not speak directly to a student’s level of cultural competence, they can inform whether various interventions or curricular or co-curricular innovations a school has implemented have resulted in more interactions among students of diverse backgrounds and can measure the perception of the school’s commitment to emphasizing interactions among students of diverse backgrounds.

With respect to commitment to pro bono, there are four questions that could be used to assess the extent to which engagements or interventions around pro bono have had a meaningful impact in the student experience:

  • How often have you participated in a clinical or pro bono project as part of a course or for academic credit?
  • Have you done pro bono work or public service?
  • How many hours do you spend in a week on legal pro bono work not required for a class or clinical course?
  • To what extent has your experience contributed to your contributing to the welfare of your community?

With respect to teamwork/collaboration, there are a handful of questions that could be instructive:

  • How often have you worked with other students on projects during class?
  • How often have you worked with classmates outside of class to prepare class assignments?
  • To what extent has your experience contributed to your working effectively with others?

Similarly, the following questions may be instructive with respect to integrity:

  • To what extent does your law school emphasize encouraging the ethical practice of the law?
  • To what extent has your experience contributed to your developing a personal code of values and ethics?
  • To what extent has your experience contributed to your understanding yourself?

As noted above in the discussion regarding cultural competence, many of these questions are not direct measures of one’s attitudes or capabilities. Rather they serve as indirect measures of how the students experience the law school culture and the extent to which that culture provides certain opportunities or experiences for students.  For law schools trying to determine whether a given change in course requirements or instructional methodologies or co-curricular programming has had a meaningful impact, these indirect measures can be a helpful supplement to other measures, such as observations of students or reflective writing.

Second, LSSSE supports supplemental question sets or modules.  This use of modules might be an even more powerful mechanism for supporting the assessment process regarding some of these professional-formation learning outcomes.  The module concept generally contemplates that multiple law schools would ask a supplemental set of questions usually focused on a common theme or attribute.  Thus, several law schools that each had a learning outcome associated with commitment to pro bono or teamwork/collaboration or self-directedness could come up with a bank of additional questions focused more significantly on one of those learning outcomes.  This “module” could then be used at each of those law schools to get feedback that is law school specific and comparative.

The most fruitful mechanism for using either existing questions or modules is to participate in multiple iterations of the LSSSE, such as bi-annually, which allows a school to see comparisons over time on two platforms.  First, one can assess the change between first-year and third-year, for example.  Second, one can assess change among first-years and third-years over time.   This can be an especially effective way to assess whether a change in curricular offerings or co-curricular programming has had a meaningful impact.

The simple point I want to communicate is that law schools need to be thinking about LSSSE as a real partner in the learning outcomes enterprise.  In addition, law schools should be thinking about working in partnership with other law schools and LSSSE on modules that could make LSSSE an even more powerful assessment tool.