Tag: Assessment

What Can Help with PBL Distance Learning?

I have received some great inquiries since I wrote my last blogpost and I’ve spoken to many teachers who have ideas and questions about teaching online with PBL math. I thought I would share some of those ideas so that everyone can benefit from these great thoughts.

Collaboration

One of the aspects of the PBL classroom that can be missing or disappear altogether is the relational aspect of discussion or collaboration. It is difficult to have students interact, especially if your school has decided to not have synchronous classes (students in different time zones, allowing families to have their own schedules, etc.). Collaborating asynchronously is difficult, but can happen. Here are some tools where this is possible:

Voicethread:
See my last post – I think Voicethread is the best app for asynchronous collaboration. It does take time to set up, but once it is, students can post the problem they have a question on and students can asynchronously post a video, audio, text or drawing question in response to each others’ comments. The thread keeps the order of the comments and the teachers can put their two cents in when necessary as well.

Flipgrid:
Some teachers that I am working with are using Flipgrid to allow students to continue doing problem partial-solution presentations and then having other student post video responses. The responses are either in the form of feedback on the video or in the form of a question or clarification. Of course, this creates a great deal of work if you are going to watch all of these videos, so you must find a way to randomly watch videos every day so the students know that you will be looking at them.

Canvas Conferences:
You might want to check whatever LMS you have, but I now that Canvas has a wonderful feature called “conferences” where you can set up a video of you chatting (it even has a whiteboard where you can write) and students can type questions. This can be recorded and posted on canvas for students that cannot be there synchronously. I guess is the same as using Zoom, but it is nice because there is communciation through the LMS automatically for the students the time and they don’t have to log onto another app.

Explain Everything Whiteboard:
If you already have an account with Explain Everything, that allows you to make videos on your iPad (and I think on your laptop too), Explain Everything also has a web whiteboard where you can create an online lesson and invite people to watch as you talk. I have not used this before but iti’s kind of an interesting add-on.

Twiddla:
This Free Online Public whiteboard is pretty cool, but it does have pop-up adds. You can make it private in the room settings and you don’t have to sign-up for an account. Of course their goal is for you to sign up for an account, so I’m sure there’s some thing in there about getting you to join, but if you just need something in a pinch, I think this is pretty great for synchronous collaboration. You can also leave anything that was written up and have kids go in later and see what was there. So it somewhat works for asynchronous as well.

Zoom Breakout Rooms:
Boy, I wish I had bought stock in Zoom about six months ago. Zoom has become the most-used app, I believe for online and distance learning in the past three weeks. I hope you are all aware of the break-out rooms that you can do in Zoom, you can triage students by putting them in these break-out rooms in groups to work on topics or separate problems as well. Of course, this is also assuming you are using synchrous time for these classes, but it a really great way to have kids collaborating in smaller groups while you can jump in between the groups – sort of like in class.

“Lessons”

I know it seems weird to think about giving direct instruction during a PBL class, but I think at this time in a crisis, it is important to think about the stress, anxiety, isolation, and other emotions that these students are feeling. All of those feelings are exacerbated in a PBL classroom so whatever we can do to help ease any of those feelings without totally moving to a direct instruction or lecture based class I think is good for the students.

One thing I have done when I teach online, especially at the beginning of the year, is try to make small videos with an app like Educreations (which giving away free Pro Accounts right now) and Explain Everything, which I use a lot (and are also giving free accounts to closed schools). These are wonderful apps and I think easy to use. Here is a video below that is a quick tutorial on how to use Explain Everything on an iPad if it would help anyone.

When I make a video to help a student(s) move forward with a problem, I try to keep it the way I would motivate their thinking in class. If they are not seeing something that is holding them back from what is needed in the problem like visualizing moving from a cone to a sector as in this video:

Another thing that I think is important is to remind students of prior knowledge that they may not see and cannot benefit from the discussion and ability of other students in the class who may have that prior knowledge more accessible to them. This is a great advantage in the face-to-face classroom that many PBL teachers rely on, is that there is usually at least one student who can say “Oh, yeah, I remember doing that before” and can give a short recall of what that prior knowledge is. Online, it is more difficult for students to access that prior knowledge that might be needed. So another type of video I make for students is a recap of prior knowledge that they might need for problems that are coming up:

Now you might be saying, “I don’t have time to make all these videos, Carmel” and that may be true. You can feel free to search on YouTube, but what I’ve found is that many of the videos on YouTube are not what i want. They are either very much based on mneumonic devices or not the type of understanding I would want my students to have. So feel free to use any of mine or email me (cschettinophd@gmail.com and I will try to make a video for you if I have time and post it on my YouTube channel.

Assessment

So it is clear that giving tests with distance learning if very difficult. Parents are not really going to sit and proctor written tests or quizzes especially if they are working from home. The idea of test integrity is also quite difficult with different age groups that you are all working with. Many students are extremely trustworthy and some schools have honor codes which kids would probably adhere to even when at home. However, the stress of being online now with a new way of learning, would be enough to make any student crack under the pressure. So here are some of the ideas that you can use to be more formative in your assessment and then perhaps a bit summative as well.

LMS Quizzes:
Many of the LMSs that schools use already have built in systems for making quick quizzes (Canvas, Schoology, Moodle, just to name a few). If one of your colleagues makes a quiz, it is very easy to share them among members of the same LMS. These quick quizzes can be open notebook and you can have 3 or 4 a week with only 3-4 questions, just to see how the students are keeping up with material. They could count towards a grade or not, but this is a wonderful way to check in. They can also be used as exit or entrance tickets.

Socrative App:
When I was still in the classroom I used this app many times in order to have a quick quiz that was open notebook and gives students real-time feedback about whether they answered a question correctly or not. You can take the time to make your quizzes or you can use the many quizzes that are already in their data base at the Socrative Quiz Shop and use those.

Summary of Topic Arcs:
I believe that writing is the best way for students to show their understanding of a topic, but writing online is very difficult. So it does make sense for students to make videos of themselves. I think a good summative assessment is for the teacher to assign to 2-3 students one of the topic arcs that has just been discussed in the recent 2-3 weeks. Students should work together synchronous or asynchronous (make a google doc perhaps) to :

  1. review and research the problems that move through that arc
  2. describe in the video how they understand those problems building up their understanding
  3. each student needs to reflect on one or two problems each and talk about how that problem is connected to the overarching theme.
  4. Use paper, a whiteboard or a virtual whiteboard to explain the problems and their understanding

The way these are graded should be up to the teacher, but what I would suggest is a collaborative grading system:

  • the 2-3 student group would get a grade based on a rubric for their own video
  • the whole class will get a single grade for the set of videos as a whole based on some basic requirements (deadlines, time limit, etc.)
  • individual students will get a grade for one feedback/question video posted for one other group’s summary video – again with a rubric so that it is clear what you are looking for
  • I will try to post some sample rubrics for this that I have used in the past

I do believe that a combination of these types of assessments will help students stress less about their grades before the end of the school year and will also help you see how much they have retained in terms of material. There is also daily contributions and completion of work that should be recorded, but I don’t think I need to tell teachers that!

I wish you all luck with this new venture and would love to hear any other ideas and your experiences with teaching PBL online.

Documents for CwiC Sessions at Anja Greer MST Conference 2016

Instead of passing out photocopies, I tried to think of a way that participants could access the “hand-outs” virtually while attending a session.  What I’ve done in the past a conferences is have them just access them on their tablet devices.  You can also go and access copies on the Conference Server if you do not have a device with you (you should be able to use your phone too).

These link to This is an Adobe Acrobat Documentpdf documents that I will refer to in the presentation about “Assessment in PBL”

Information on Spring Term Project and Spring Term Project Varignon 2015 (this document includes rubric)
Keeping a Journal for Math Class
Revised Problem Set Grading Rubric new
Rubric for Sliceform project and Sliceforms Information Packet
Weekly-Learning-Reflection-Sheet

Page at my website with Rubrics and other guides for Assessment

Someday I’ll get this assessment thing right… (Part 2 of giving feedback before grades)

3 Comments

So, all assessments are back to the students, tears have been dried and we are now onto our next problem set (what we are calling these assessments).  What we’ve learned is that the rubric allowed us to easily see when a student had good conceptual understanding but perhaps lower skill levels (what we are used to calling “careless mistakes” or worse). We could also quickly see which problems many students had issue with once we compared the rubrics because, for example, problem number 6 was showing up quite often in the 1 row of the conceptual column.  This information was really valuable to us.  However, one thing we didn’t do was take pictures of all of this information to see if we could have a record of the student growth over the whole year. Perhaps an electronic method of grading – a shared google sheet for each student or something to that effect  might be helpful in the future – but not this day (as Aragorn says) – way too much going on right now.

We also changed the rubric a bit for a few reasons.  First, we found that when students completed the problem to our expectations on the initial attempt we felt that they should just receive 3’s for the other two categories automatically.  We considered not scoring them in this category but numerically felt that it was actually putting students who correctly completed a problem at a disadvantage (giving them fewer overall points in the end). Second, we also changed the idea that if you did not write anything on the revisions you earned 0 points for the revisions columns.  Many students told me afterwards that they felt like they just ran out of time on the revisions and actually had read the feedback.  This was unfortunate to me since we had spent so long writing up the feedback in the hope that the learning experience would continue while doing revisions.

Here is the new version of the rubric: Revised Problem Set Grading Rubric new

What we decided to do was to try the revisions this time without the “explanation” part of writing.  I think it will keep the students focused on reading the comments and attempting a new solution.  I was frankly surprised at how many students stuck to the honor pledge and really did not talk to each other (as they still got the problem wrong the second time around – with feedback).  Truly impressive self-control from the students in my classes and how they were sincerely trying to use the experience as a learning opportunity.

I do think the second assessment will go more smoothly as I am better at doing the feedback and the rubric grading.  The students are now familiar with what we are looking for and how we will count the revisions and their work during that time.  Overall, I am excited about the response we’ve received from the kids and hope that this second time is a little less time-consuming.  If not, maybe I’ll just pull my hair out but I’ll probably keep doing this!

 

 

 

Why Teachers Don’t Give Feedback instead of Grades, and Why We Should

7 Comments

First in a series of posts about my experiences with “Feedback Before Grades”

Holy Mackerel is all I have to say – Ok, well, no I have plenty more to say – but after about a week and a half of holing myself up with my colleague, Kristen McVaugh, (big shout-out to Ms McVaugh who is not only teaching PBL for the first time but was willing to dive into this amazing journey of alternative assessment with me this year too), I am totally exhausted, almost blind as a bat, partially jaded and crazy – but mostly ready for a drink.  This little looped video of Nathaniel Rateliff and the Night Sweats pretty much sums it up…

So here was our well-intentioned plan:  we wanted to start the year off with a different type of assessment.  I put out my feelers on twitter and asked around if anyone had a rubric for grading assessments where the teacher first gave only feedback and then allowed students to do revisions and then once the revisions were done the students received a grade. Kristen and I knew a few things:

  1. we wanted to make sure the revisions were done in class
  2. we wanted to make sure the revisions were the students’ own work (tough one)
  3. we wanted to give students feedback that they needed to interpret as helpful so that we weren’t giving them the answer – so that it was still assessing their knowledge the second time around
  4. we wanted to make sure that students were actually learning during the assessment
  5. we wanted students to view the assessment as a learning experience
  6. we wanted students to be rewarded for both conceptual knowledge and their skills in the problem solving too

So we created this rubric Initial Draft of Rubric for Grading.  It allowed us to look at the initial conceptual understanding the student came to the problem set with and also the initial skill level. Kristen and I spent hours and hours writing feedback on the students’ papers regarding their errors, good work and what revisions needed to be done in a back-handed sort of way.

Here are some examples:

Student 1 Initial Work

Student 2s initial work

Student 3 initial work

 

Some kids’ work warranted more writing and some warranted less.  Of course if it was wonderful we just wrote something like, excellent work and perhaps wrote and extension question.  The hard part was filling out the rubric.  So for example, I’ll take Student 3’s work on problem 6 which is the last one above. Here is the rubric filled out for him:

Student 3’s Rubric

You will notice that I put problem 6 as a 1 for conceptual understanding and a 2 for skill level (in purple). In this problem students were asked to find a non-square quadrilateral with side lengths of sqrt(17).  Student 3 was definitely able to find vertices of a quadrilateral, but he was unable to use the PT to find common lengths of sides.  I gave him feedback that looking at sqrt(17) as a hypotenuse of a right triangle (as we had done in class) would help a bit and even wrote the PT with 17 as the hypotenuse in the hope of stimulating his memory when he did the revisions.

The day of the revisions Student 3 was only capable of producing this:

Student 3 revisions

He followed my direction and used 4 and 1 (which are two integers that give a hypotenuse of 17, but did not complete the problem by getting all side lengths the same. In fact, conceptually he kind of missed the boat on the fact that the sqrt(17) was supposed to be the side of the quadrilateral altogether.

 

One success story was Student 2.  She also did this problem incorrectly at first by realizing that you could use 4 and 1 as the sides of a right triangle with sqrt(17) as the hypotenuse but never found the coordinates of the vertices for me. I gave her feedback saying there might be an easier way to do this because she needed vertices.  However, she was able to produce this:

Student 2s revision
Student 2s revision

Although she did not give me integer-valued coordinates (which was not required) and she approximated which officially would not really give sqrt(17) lengths it came pretty darn close! I was impressed with the ingenuity and risk-taking that she used and the conceptual knowledge plus the skill-level. Yes, most other kids just used some combination of 1’s and 4’s all the way around but she followed her own thought pattern and did it this way.  Kudos to student 2 in my book.

Next time I will talk about some of the lessons we learned, other artifacts from the kids’ work and what we are changing for next time! Oh yeah and some great martini recipes!

Buyer Beware…when using rubrics for critical thinking skills

One of my goals in my work is often to help classroom mathematics teachers to be more deliberate in the ways in which they assess problem solving.  Although many people can be cynical about rubrics, I think that students can find them at least helpful to know what a teacher expects of them.  I have some students who told me that they pull out my rubric for grading journal writing almost every time they go to write a journal entry this fall.

However, a rubric that is vague and ambiguous about expectations can cause more harm than good.  Just throwing a rubric around that students can look at, or one that you can post on your website that you can show an administrator and say, “See, I have a rubric for that” isn’t necessarily a good thing.  Especially for problem solving.  Problem solving as a process is a very difficult thing to nail down for students especially in terms of the levels of how they can improve in their work.

I recently ran across this rubric that posted on a website under the title “Awesome Problem Solving Rubric for Teachers.”

Is this an “Awesome Rubric” for teachers?

As I read through this, at first glance the categories look pretty good – Identify the problem, identify relevant information, analyze the problem, use strategies and reflect on the process.  Sounds like a pretty standard problem solving process –very similar in many ways to Polya’s process or the steps that Jo Boaler discussed in her online course How to Learn Math this summer.

The graded level descriptors of how a student might be able to see where their work “fits” in the rubric seems to only put the behaviors on a “continuum” of Always- sometimes- never instead of trying to describe actions that the student could do that describe a mediocre way of using a strategy.    For example, analyzing a problem can be so much more descriptive than just “I think carefully” about the problem before a student starts.

They could:

1. listen deliberately to others’ ideas and reflect on them in writing or verbally

2. question the given information of a problem – does it make sense in a realistic way?

3. think about the representations they can come up with for the problem – does a graphical approach make the most sense?  Why?  Would making a geometric representation be better, if so why?

4.  In comparing a new problem to ones I’ve already done, can I list the similarities and differences?  What is this question asking that others I’ve done not asked?

How many students can really ascertain what “thinking carefully” about a problem is?  I have found that more and more we need to erase as much ambiguity as possible to help students learn to be critical thinkers.  As we feel the need to teach critical thinking, reasoning skills and sense making, it is even more imperative to have rubrics that are as precise as possible.

Now, I don’t claim that mine are perfect, but my rubrics and student feedback forms have gotten some pretty good reviews from teachers and successful feedback from students.  I work on them every summer and am continually editing in order to be more deliberate about the feedback I give my students.

I also highly recommend the rubrics from the Buck Institute Website under their “tools” category.  I also adapted one of their critical thinking rubrics that was aligned to the Common Core and changed it directly for my PBL curriculum – more for presentation of problems and novel problem solving.  I’m still working on it because I have to think about exemplars for what would be above standards, but let me know if you have any feedback.

Critical Thinking rubric for PBL

So, I would just warn anyone to beware of “awesome rubrics” for teachers that they find on the internet because something that might seem awesome at first glance might end up doing more harm than good.