Review of Content Management Systems Through the Lens of Assessment

This week in CEP 813 we examined and compared the affordances of free Content Management Systems (CMSs) through the lens of assessing student learning. I chose to compare Weebly for Education, COURSEsites by Blackboard, and Haiku Learning. The results of my critical review of the three sites listed can be found in this spreadsheet.

Based upon my analysis of these three CMSs, I have chosen to use Haiku Learning to create my CMS assessment. The main reason I chose Haiku Learning is because it not only had all of the redeeming qualities and capabilities I was looking for in the assessment framework, but it also was very user friendly- from an instructor and student perspective. All three sites appeared to offer unique and differentiated options for assessing students-formative and summative, but I was most impressed with the design and usability of Haiku Learning. For example, Haiku Learning offers the following options that could easily be used to assess student learning: 1) discussion forums, 2) dropbox submission, 3) surveys, 4) rubrics, 5) learning portfolios, and 6) peer and self assessment tools. However, I did find that COURSEsites by Blackboard offered perhaps more specific options for collaborative group work, like assigning roles and jobs, but it was much more difficult to navigate and set up than both Weebly for Education and Haiku Learning, which made the features not worth it to me. I felt like Haiku Learning‘s traditional and non-traditional assessment tools worked harmoniously within the framework design and flowed well with content, opposed to Weebly for Education and COURSEsites by Blackboard where I spent a lot of time clicking around in different places to utilize and create different assessment types, but again, that is a design issue not an issue of whether or not the features exist. I am just a huge believe in USEFUL and USEABLE tools, and the challenging design framework in both Weebly for Education and COURSEsites by Blackboard made some of the features and tools less USEABLE.

Since assessment should inform teaching and learning and in many ways drive our instruction, I was pleased to see the various options for feedback in relation to the assessment type. Some sort of gradebook feature is available on each of the CMSs I reviewed, keeping both parents and students informed and up to date. Additionally, through various multimedia tools and options on each of the sites, students are able to express themselves creatively and receive direct feedback via comments from their instructors and peers. For example, in math class, I like to utilize GeoGebra as a teaching, learning and assessing tool. Unless I used a link to a new page for each GeoGebra activity, I would need to be able to embed Java applets into my site. Having learned to navigate Weebly for Education quite well over the past few years and through my explorations with Haiku Learning the past few weeks in both this course and CEP 820: Teaching Students Online, I know I am able to embed Java applets into both of these sites; however, I was not able to figure out how to embed Java applets into COURSEsites by Blackboard. I don’t want to say it isn’t possible, though, because I am still learning to navigate the site and it’s tools. To view a GeoGebra lesson I created on Weebly with a Java applet embedded in the site page, check out my webpage on quadrilaterals and technology. Feel free to play around with the Java applet 🙂

Another great feature I was surprised to find is that all three sites offered an equation editor. As a math teacher, that was a feature I looked for right away. Having created online assessments using a shifty equation editor in Schoolnet for over three years now, I have become all too familiar with writing an equation in Microsoft Word, taking a screenshot of the equation, uploading the PNG file to the assessment question, and then repeating for additional assessment questions as needed. This daunting process is something I would definitely want to avoid while developing an online math course because we use equations so often in math class. As far as I can tell, the equation editors in each of the three sites appear to be up to par and don’t require coding like the equation editor in Schoolnet does.

Moving forward, the plan is to create a collaborative formative assessment activity using Haiku Learning and GeoGebra that allows learners to explore the Pythagorean Theorem. My Math 8 students will work together in a collaborative group space, using GeoGebra to explore and develop an informal proof of the Pythagorean Theorem. Through this activity I will be able to monitor and direct learning by providing feedback and stimulating discussion conversation, allowing my learners to eventually demonstrate their understanding of the Pythagorean Theorem.  In the assessment, students will go on to become “Problem Writers” posting a real-world question involving the Pythagorean Theorem for their peers to read. Then, they will go on to review and respond to at least three questions posted by their classmates.  This will foster collaboration between the students while also assessing students’ understanding of the Pythagorean Theorem (the question they write) and it’s application (their ability to solve their classmates real-world questions).


CEP 813 – Module 1 – Annotated Assessment/Evaluation Exemplar

In Module 1: Foundations of Assessment and Evaluation, we were asked to take a critical look at the design of a typical classroom assessment we have used in our teaching profession. The assessment I am analyzing was designed as a summative assessment for an 8th grade math unit on the Pythagorean Theorem. The formatting/sizing of the questions may look funky because I built the test on Schoolnet, an online assessment and reporting system we use in my school district, so it is a digital assessment that I downloaded into a PDF version so you all could see.

Underneath the assessment shown below, you will find my critical analysis of the assessment.

Critical Analysis of Assessment Design:

a) How would I describe the design of this assessment? The assessment was designed as a summative (end of chapter test) for an 8th grade math unit on the Pythagorean Theorem. The assessment includes multiple choice, gridded (type in the number), and open response questions. The assessment is also designed to have at least a two question spread for each of the six skills (3 standards) I assessed.

b) What is the purpose of the assessment? Since it is a summative assessment, it is designed to be evaluative rather than diagnose. That is, the purpose of this assessment is to determine levels of understanding and achievement for each learner and convey that information to them in a transparent way, evaluate the effectiveness of my instruction and assessment design for the unit and make changes accordingly, measure each learners’ progress/improvement on each learning goal (skill in terms of growth and proficiency levels, and make decisions regarding students that need re-teaching and re-assessment before moving onto the next unit. Specifically, the purpose of this assessment is to convey the growth and mastery level of each skill for each student in a transparent way and report/record the achievement levels. The specific learning goals/standards are shown in part (c).

c) How does this assessment align, if at all, with the curriculum standards that guide my professional practice?  The assessment aligns with the Common Core State Standards for 8th Grade Mathematics. There are three standards that align with the unit, which are broken down further into six skills that can be seen in the document shown beneath the standards.

CCSS.Math.Content.8.G.B: Understand and apply the Pythagorean Theorem.

  • MATH.CONTENT.8.G.B.6: Explain a proof of the Pythagorean Theorem and its converse.
  • Skills for standard:
  • MATH.CONTENT.8.G.B.7: Apply the Pythagorean Theorem to determine unknown side lengths in right triangles in real-world and mathematical problems in two and three dimensions.
  • MATH.CONTENT.8.G.B.8: Apply the Pythagorean Theorem to find the distance between two points in a coordinate system.

d) What information will this assessment give me about each student? This summative assessment will tell me each students scale score for each skill assessed this unit. That is, I will know the level of understanding each student has in relation to each skill I assessed. I will also look for growth shown throughout the unit. Between smaller assessments covering each individual skill (concept quizzes) and formative assessments in class, the students and I have been tracking their progression throughout the unit. The summative assessment should reveal growth for each student from where they were at the start of the unit, where they were as we progressed in the unit, and where they were when they finished the unit.

e) How do I intend to use the information provided by this assessment? I use the information on summative assessments to tell me which students need to stay after for re-teaching sessions and a re-assessment. Students only re-assess on skills they aren’t proficient in, so they aren’t necessarily retaking the entire assessment. The nice thing about having concept quizzes and formative assessments throughout the unit is that most of the misconceptions and/or gaps in understanding have been addressed before we get to the summative assessment. Like I said previously, the summative assessment is evaluative rather than diagnostic. I typically have a good idea on what students know and how they are going to do before they take the summative assessment. I think the students would agree and say they also have a pretty good idea of how they will perform on the summative assessment. I use the findings from the summative assessment to report progress and achievement to students/parents in a transparent way. Further, I use the data collected on summative assessments to analyze, modify, and improve the assessment. Sometimes the data reveals that I may have a confusing or just plain bad question that needs to be thrown out or rewritten, so I use those findings to evaluate the effectiveness of my assessment, too.

f) What assumptions have I made about whether this assessment will, in fact, give me the information I need about the students who do it? I have assumed that this assessment will reveal accurate results regarding each student’s level of proficiency in each of the six skills. I have assumed that my questions are transparent, accurate, effective, and do not need to be modified. I have assumed that the scale scores I have established are accurate, fair, and represent growth and achievement levels accurately for each learner & each skill. I have assumed that students who practiced the skills and made improvements based on the feedback they received throughout the unit will continue to improve and will show levels of mastery for each skill being assessed. Contrastingly, students who have continuously not completed practice sets, concept quizzes, or participated in formative assessments in class assessments will lack growth in most areas and will not show levels of proficiency for most, if not all, of the skills being assessed. I have assumed that this assessment design will accurately assess all of the skills we have covered this unit. Lastly, I have assumed that the assessment can be easily broken down into the different skills being assessed to show where students need improvement and where they are proficient.

g) What skills have I assumed students have that will enable them to complete this assignment? I have assumed that students can read and work technology, since the assessment is online. I have also assumed that they understand how to use the online assessment and the embedded tools. In relation to the content, I have assumed that, when given formulas, students can perform mathematical operations to either simplify or solve equations depending on the scenario… although I am not sure I would say I assumed that information since this is the summative assessment and I typically have a pretty good understanding of what students can and cannot do mathematically at this point.

h) For whom would this assessment prove difficult? Why?  This assessment will be difficult for students who struggle with application of mathematical concepts and theorems in real world and mathematical scenarios. Moreover, students that struggle with relating mathematical formulas to word scenarios will struggle as several questions require deconstructing text and applying multistep mathematical computations. Students who struggle with graphing will also struggle with the skills that requiring graphing distances in the coordinate plane. Further, students who have not practiced the skills throughout the unit or worked towards improvement based on the feedback they received throughout the unit will also struggle on the summative assessment.

i) Based on my readings this week, are there ways that I can imagine re-designing this assessment so that it’s better in some way? Explain your rationale and justification for your re-design idea(s). I think there is always room for improvement, and after this weeks readings, I definitely think this assessment should be re-imagined and structured to motivate more students to want to do well rather than complete it as a means to an end. I think including a reflective piece or maybe a short answer portion where students write their own problem would be one way I could attempt to improve the design and learner motivation. Another idea would be to include some sort of project or performance task where learners complete the proof of the Pythagorean Theorem using manipulatives. Although, that is already a formative task we do earlier in the unit. I do think that some sort of hands on task would be a way to reach learners who are better at orally explaining what they know or physically showing what they know.

I think that the assessment I made is designed well in relation to the content and skills I want to assess and in terms of the information I hope to get form the assessment. Prior to this point there have been several formative assessments embedded within instruction and feedback has been provided several times daily, so my students have been able to diagnose weak areas and improve them before the summative. Perhaps, instead of giving the assessment online with all of the skills on one test, I could break the summative into six separate assessments so the learner has a clearer understanding of what skill is being assessed, although, reasoning to decide what the actual problem is or what the question is asking and then deciding how to complete the task is an important feature of the design of the summative test that covers all six skills. I think that separating the skills into separate assessments may trivialize the assessment all together. Part of what I am looking for is whether students know when to apply the Pythagorean Theorem or its Converse and whether or not they know which method, formula, and/or theorem they need to complete real world and mathematical tasks.