Assessment

SENCER SALG

Student Assessment of their Learning Gains

The SENCER Student Assessment of their Learning Gains (SALG) allows students to rate how well specific activities in SENCER courses help their learning. The assessment tool also asks students to report on their science skills and interests, as well as the civic activities in which they engage.

The primary purpose of the SALG is to provide useful, formative feedback to faculty interested in improving their teaching. Students rate how well class activities such as lectures, discussions, or labs help their learning. The SALG also provides a snapshot of student skills and attitudes at the beginning and end of courses, allowing instructors to gauge the effectiveness of their instruction in specific areas. The SENCER-SALG also informs the national assessment of the SENCER program.

The SALG is unlike the traditional Faculty Course Questionnaire in that it does not ask students to rate the competencies of their instructors. It is also not meant to be used as a test or quiz. In the spring of 2008, a new, updated SALG platform was launched--the result of efforts by a team supported by the National Science Foundation. Many improvements have been made to the instrument and the website to ease use and to take advantage of the latest research on teaching and learning.

The SALG retains its basic format of 10 question stems: six related to course design and practices, and four related to course learning objectives. To ensure the conceptual identity of the SALG, these 10 question stems can no longer be changed or deleted, nor can their answer scales be altered. Within these limits, however, you may add to, delete, or change any of the sub-questions under these stems to adapt your SALG to the specific design and goals of your course. These sub-questions are now grouped more coherently, and their content has been updated to apply to a broader variety of disciplines and course types.

As before, you can also choose to begin from an instrument you have used previously or from an instrument developed by a colleague. The new site allows you to merge instruments easily, so that you can take advantage of an instrument that you have developed for a particular course, and still easily incorporate a whole set of questions present in another instrument--your own or someone else’s. Finally, the new site includes more powerful analytical tools to help you better understand the feedback gathered by your SALG, including a tool that allows you to code students' answers to open-ended questions so that you can analyze them statistically.


Instructions for Creating a SENCER-SALG Instrument

  1. Open a web browser. Firefox and Internet Explorer are preferred. Chrome usually works decently. Please do not use Safari, it is not compatible with the SALG software.
  2. Go to www.salgsite.org.
  3. Log in, using your (school) email address and password. (If you have not logged into the site before, you should create an account using the email address you gave to SENCER.)
  4. On your home page, below the soft-cornered rectangle with your department affiliation in it (and above the list of your instruments), you will see two large buttons. Click on the right button, “Implement an instrument created by your department without adding questions.”
  5. The next window should open a list of all department instruments you are eligible to use. Find “SENCER-SALG 3: An end-of-term instrument for all SENCER courses” in the Description column and click on the “Select” button next to that description.
  6. The next page, “Wizard: Preview Instrument,” contains the SENCER-SALG instrument. Confirm that this is the appropriate instrument by checking question 1.5. It should read, “The power of science as a problem-solving system to affect my life.”
  7. Click the “Next” button at the top of the page.
  8. Set the open and close dates for your survey.
  9. DO NOT SET THE OPEN DATE TO TODAY. Set the open date to sometime in the last week of classes or in finals week. You will need some time to edit the questions.
  10. Set the close date to a week or two after the open date to give students adequate time to complete the survey. (If you are planning to require the survey or to give extra credit for completing it, you will want to set the close date to a date before grades are due.)
  11. Write down the instrument near the top of the soft-cornered rectangle; you’ll need it later.
  12. Click Next.
  13. Choose the announcement you will use to announce the survey to students. When you click on the link, your email client should launch, and you should see an email formatted and ready to send to your students. You will have to paste in the list of student email addresses yourself. You can choose to send this email now, or you can wait to send it until you have edited the questions.
  14. Tip: I prefer to wait until I have edited the questions before I send this email because students often lose the email if they can’t do the survey as soon as they get the email.
  15. Tip: I find it convenient to change the close date on the email to a date several days earlier than the actual close date. That way, when students forget to do the survey by the due date, I can let them know they can still complete it without having to open a new survey. (Once the survey closes, you can’t reopen it, because allowing that could potentially compromise student anonymity.)
  16. Click on Done. Click on Done again on the next page.
  17. You are then returned to your home page. Scroll down the page to find the instrument you just created. You can identify your instrument by the number you wrote down in step eight. You can also find it by looking for “SENCER-SALG 3” under the Course column.
  18. Click on the number of your instrument in the far left column.
  19. You will be taken to the dashboard for this instrument.
  20. Click on “4) Edit the questions.”
  21. On the edit page, you can make adjustments to many of the questions. (Some questions are locked because they collect data critical to the final report to the NSF and other funders.)
  22. You may delete any question with a delete button in the right column.
  23. You may move questions by clicking on the up and down arrows next to the delete buttons.
  24. You may edit any question with a green “edit” button next to the text.
  25. You may add questions within any section by choosing “Add sub-question.” (This button is located at the top of each section.)
  26. You may also add entirely new questions by clicking on the “Add question group” button at the bottom of the survey.
  27. If you are making more than a few changes, SAVE YOUR WORK OFTEN! (Save your work by going to the bottom of the page and clicking on “Save and continue.”)
  28. When you are finished editing, click on “Save and done.”
  29. When you are finished editing and are sure that the survey is ready to send,
  30. You can click on “5) Edit the start and end dates for your survey” and set the start date to tomorrow.
  31. You can click on “6) Announce the survey to students.” Follow the directions for step nine above.
  32. Congratulations, you have finished creating your SENCER-SALG survey.
  33. After the close date of your survey, you can check the results by logging back in, selecting your survey (by clicking on the number of the survey), and choosing “7) Analyze and/or download the results of your survey.”

Website
http://salgsite.org
Go to the SALG page to learn more, read validity studies, and create an instrument for your course.

Questions
Please contact Stephen Carroll at scarroll[at]scu.edu if you are having trouble setting up or using a SENCER SALG instrument.

The Scholarship of Teaching and Learning

The Scholarship of Teaching and Learning (SoTL) is a field dedicated to engaging in teaching as a scholarly pursuit. Many SENCER faculty find it useful to use the SoTL method when redesigning their courses to ensure that their changes are advancing student learning. SoTL also offers an avenue for publication on lessons learned, and fosters the sharing of strategies and techniques proven to be effective.

In May 2016, NCSCE hosted a webinar on using the Scholarship of Teaching and Learning to tell the story of curriculum development. The webinar provided an overview of the key aspects of SoTL, situating it within a spectrum of scholarly work on teaching and learning. Two in-depth case studies, one involving service learning and the other involving sustainability, illustrated how SoTL can contribute to the process of developing, assessing, and disseminating curriculum. One particular SoTL component highlighted was the role literature searches play in both shaping and refining questions as well as providing the background context required for publication. To learn more, please view the recording of the webinar.

Evidence Matters SoTL Webinar

Dr. Matthew A. Fisher of Saint Vincent University has also presented extensively on using the SoTL method. You can watch a recording of his "Inquiring Into Our Students' Learning - The Scholarship of Teaching and Learning" presentation, which provides helpful background on SoTL.

Setting Learning Goals in SENCER Courses

Good assessment starts with effective goal setting, yet the challenge of developing measurable learning goals eludes many course developers. The SENCER Summer Institutes emphasize effective goal setting as a major program challenge. NCSCE Senior Fellow Barbara Tewksbury, Upson Chair of Discourse at Hamilton College, serves as principal investigator on a major NSF project, "On the Cutting Edge." Her plenary presentations at SENCER Summer Institutes introduce members of the SENCER community to the art of setting learning goals.

The SENCER Rubric 2.0

What makes a course or curricular project a “SENCER course” or a “SENCER project”? To what extent does an examination of a course or project demonstrate the presence or absence of components associated with the SENCER Ideals? These are the two basic questions that the rubric is designed to help answer.

Various elements of course design, faculty practice, and institutional policy making--some common to good designs and practices and others more specific to the SENCER approach--are presented in the following pages, each with a brief description. For each, those completing the rubric are invited to consider the following four options:

  1. Not Observed - the element was not observed in the material reviewed;
  2. Basic - the review showed evidence that the element was present at a level described in the chart as “basic”;
  3. Advanced - the review showed evidence of the presence of the element at a level described as “advanced”; and
  4. Transformative - the review showed evidence that was so advanced so as to be transformative, according to the application of the rubric.

Using the rubric is like doing an audit; that is, you will be looking at material evidence to make the assessments. The evidence may consist of a review of relevant course materials, such as syllabi, texts, websites, assignments, completed projects and tests, assessment findings, video/audiotapes, reports, and journals. The evidence may also come from transcripts of interviews with students and professors, etc. When using the rubric, you will want to note your “findings” as well as the source of the evidence on the rubric form itself in the space below each “table.”

SENCER Rubric 2.0

Download (PDF, 106KB)

If you have questions regarding any of these tools, please contact:

SENCER-SALG
Stephen Carroll
scarroll[at]scu.edu

On the Cutting Edge
Barbara Tewksbury
btewksbu[at]hamilton.edu

SENCER Rubric 2.0
Wm. David Burns
david.burns[at]sencer.net