Sunday, April 2, 2017

A Little Knowledge is...

If you read my last post, you know I was feeling good about filling some gaps in my knowledge around use of Conestoga College's new CLB-aligned rubric templates. Well, my optimistic feelings about this tool have now been dampened a bit as I sit here looking at a stack of finished marking and the results of a reading assessment I gave to my seniors' class this week. It's a multilevel class to whom I usually give CLB 3 material, not at all challenging to the higher level students. This week was one of those times when I cater to their needs and teach a bit higher.  The module went very well; the reading assessment did not. If anything, the level 2/3 students did as well or better than the 6/7s.

When students who I KNOW have CLB 5 or higher reading skills get 30% on my test, I know that the problem wasn't with them. The problem is that I have no business designing and scoring such tests when I have only had a few hours of instruction (workshops and TESL training combined) in this sub-specialty of ELT: test design.

I am not using materials / texts that have been graded. I'm using authentic texts such as the guide to eating safer fish from the Detroit River, which is immediately relevant to my learners' lives.  Once upon a time long, long ago I ran across a tool that allowed me to paste in text, click a button, and receive analysis of the level of text based on how many words came from each tier of the first x thousand words language learners understand. If I ever find it again, or if you do, let's remember to link to it on Tutela or here, shall we? In the meantime, I gave it my best guess. What about my test questions? I THOUGHT I had used phrasing that was easy to understand, AND we went over the questions quickly the day before the assessment. Only when I marked the tests and saw that many students misinterpreted the same questions did I start to see the flaws in my design. Test designers have the benefit of being able to pilot their tests before live roll-out. My poor students are continual guinea pigs.

I will explain all this to them, apologize sincerely, and do what next time? I don't know yet.

In the meantime, I am starting to become privy to evidence, anecdotal though it may be, that my agency is taking this whole PBLA thing a lot more seriously than many other schools are. Have you yet started to receive students from schools that are supposedly already doing PBLA? What do these students' binders look like, if they bring them at all? Are they filled with spelling and grammar tests instead of skill using and real-world tasks? Have they even had the plastic taken off the guts? Has the other school entered into our shared database the fact that the student was issued a Language Companion?

There was a time not so very long ago when the teachers at my agency had a word-of-mouth reputation so strong among newcomers that our waiting lists were the longest in the city. Now? I fear some are leaving us in favour of schools that are "softer" on PBLA.

If this is indeed the case, then I am ready to consider mandatory use of ePortfolios whereby all agencies mandated to implement PBLA also be required to use the SAME electronic portfolio system. When a student transfers, she brings the password to her ePortfolio with her. We can see which school issued her LC, the name of the instructor who last promoted her, the work on which that promotion was based.

Without some sort of uniform and consistent accountability for schools and instructors, this whole PBLA initiative is nothing but a contest to see which agencies can fudge the best. Sorry if I sound peeved, but I am. I feel as if good teachers and schools are being punished for true compliance while those not taking it seriously face no repercussions. If that is the case, what's the point? It also feels as if good, passionate teachers are getting snagged in a dragnet that was initially created to catch out the lazy and incompetent.

That's my rant for the month. By next week I hope to have lots of juicy free stuff uploaded to kellymorrissey.com from my seniors' module on safe fish, my literacy module on Canada, and the seniors' module on using 211. Let's hope!

12 comments:

  1. Interesting reflection and I am totally on board with the e-portfolio idea. I teach a number of students who study at two schools. Some do have a binder and the other school has implemented PBLA. However, there are other students who attend another school in addition to where I teach and PBLA is not being implemented. There is no way for me to see what they are producing or how they are performing at this other school. An electronic database makes much more sense to me and I do float this idea to my superiors quite often. I live in hope and to trust that your discussion of this past assessment experience will bring about helpful learnings for both you and your students.

    ReplyDelete
  2. I wonder if these are the sort of tools you mean to "grade" texts. I went to Douglas Orme's workshop at TESL toronto's 2015 T4T conference. Interesting.
    http://textanalysistools.pbworks.com/w/file/fetch/92090808/TAT4T-Jan31-2015.pdf
    Younare right on the money realising that teachers (who are not trained test creators) being asked to create assessments leads to questions of reliability, validity etc. I've already seen stuff that made my hair stand on end. I imagine in different times you would have created a comprehension and inference quiz for the class from the realia - and some local lore would have been picked up, some vocabulary and a general awareness of issues of fishing in polluted waters (Earth Day coming up) as well as "careful reading" strategies. LOL - when I saw your "rubric" (and I recognised the "model" it followed) I thought to myself - hmm - real life task? Shouldn't students have actually caught, cooked and eaten fish to move to next Benchmark?
    I can't agree with the idea of "electronic databases" or "mandatory use of e
    Pottfolios" just because you have to fulfill requirements that others don't! What is this - equal suffering for all!!! I'm rather enjoying the Schadenfreude that comes when I hear of the slew of impractical requirements that some are forced to perform and I am full of admiration and respect for those managers/administrators in the Boards who are using common sense - and putting the needs of their clients and their teachers first (not the needs of the designers of the project).It is the "mandatory" part of PBLA that is so egregious, and the "one size fits all" assumptions that are so misguided.
    Sigh. A healthy "cottage industry" has already sprung up around PBLA creating yet more "resources" that cost $$$$ and will not be used. Did Conestoga get paid for the "rubrics creator" (migrated CLB material on a "sort of" rubric format)? How will the ROI be measured?

    ReplyDelete
    Replies
    1. Thanks for the link! That is exactly the type of tool I meant. Equal suffering for all! Lol. You caught me being an second grader emotionally.

      Delete
  3. Has anyone considered petitioning MCI in an attempt to persuade them to drop this PBLA nonsense? Teachers are having borderline nervous breakdowns over their complete inability to keep up with marking. Students are not learning to speak ENGLISH which is what they desperately need and want. Instead they spend hours upon hours learning to organize binders - dated old fashioned binders which are also full of govt info which the students could google to find out. Most ESL students have smart phones, tablets, laptops and can find out whatever they need with a click of the mouse as opposed to poring through pages and pages of old timey papers and bulky binders which also take up much needed space in classrooms because who wants to lug home a great heavy binder full of papers? Earth Day is coming up... talk about waste. Classes are overflowing with a backlog of students who cannot move to the next level because teachers can't keep up with the marking of their tasks, and until they complete all of their tasks, they're not going anywhere. Their futures are being held up - can't move on - can't do credit courses - can't get a high school diploma - can't take courses until they finish ESL but they can't finish because the teachers are too swamped with marking to be able to keep up. Has anyone thought of organizing a petition or inviting MCI to actual schools to see the actual disastrous results of this? Is anyone thinking of going on strike? That of course, is unrealistic because most ESL teachers don't make enough money to be able to afford the luxury of a strike. We are not salaried (is anyone?) I heard this Monday in our staffroom that one woman spent her entire weekend - until midnight Sunday - marking papers.
    So - portfolios - because we are powerless to change this. Portfolios to fulfill requirements... paying lip service... we have students who want to learn to speak ENGLISH!!!! We have teachers who know how to deliver and be flexible and adapt and help them but their learning is all about organizing binders now. I suppose the govt won't back down now that they've wasted all of this money. I do not know one teacher - not one - who thinks that PBLA is a good idea. It makes me sad.

    ReplyDelete
    Replies
    1. interesting thiught - petitian MCI. I heard at the scuttlebutt that some elements in ESL in Ontario are none too happy with the Feds dictating what should be the overriding methodology in Ontario - why should they be?Ontario has its own approach and projects which were not designed to "align" with the "one size fits all" PBLA approach. Of course Ontario wants to mprove ESL program delivery but not necessarily through the mandated untried, new "learning assessment methodology " that is PBLA.
      But this issue is complicated because the Government of Ontario has given milllions of dollars to the Centre For Canadian Language Benchmarks which MANAGES PBLA. I calculated $16 million over the past four years (from annual CCLB reports at language.ca) This is by far the largest contribution of all the provinces..How much goes to PBLA and how it is spent I cannot find out. Perhaps it is time to ask the government of Ontario for an accounting of the money sent to CCLB - and an explanation of how they will measure the return on investment. Or if the government won't tell us then perhaps the opposition could find out. Or the Auditor General?
      Every time I hear "The funders want it" my blood boils. Moi, Ms Taxpayer am the funder....show me proof my money is being well spent and I will shut up.

      Delete
    2. Anonymous: You bring up so many excellent points! I wish there were a way for me to carry your words to the decision makers. Petition? I don't know how that would work when so many fear reprisal, feel they can only speak out under the cover of anonymity. We're in a sad fix, indeed. K

      Claudie, thank you for reminding me that IRCC is not the funder. We are. K

      Delete
  4. This comment has been removed by the author.

    ReplyDelete
  5. IF this becomes an electronic file that can be passed form colleague to colleague for criticism for assessment I will die.

    ReplyDelete
  6. Kelly, thank-you again for your openness in sharing your materials, your ideas and sometimes your struggles. I teach LINC and I also question whether my PBLA tasks are really measuring what they are supposed to and what "adequate control" really means.
    Are you thinking of LexTutor's Vocab Profiler in your post? Or did you find a more precise tool? The Vocab Profiler measures the 1st and 2nd 1000, the AWL, and excluded words. (Tom Cobb has also added a "Compleat Web VP">)
    I agree that the portfolio binders are not serving their intended purpose and, despite what the PBLA materials say, students do NOT want to take their binders home every night and bring them back again the next day. I've used eportfolios with CLB 6 students. It make much more sense for everyone but right now, it's a lot of extra work for the instructor. I know IRCC was looking at eportfolios a year or so ago but I've heard nothing since then.

    ReplyDelete
    Replies
    1. Yes, Bonnie! That's the tool I was thinking of. I'm embarrassed to say that when I just visited my own classroom blog to add this link to the "FOR TEACHERS" section of the sidebar, I found that I had already included it years ago. Doh!

      Delete

Thank you for participating in this forum. Anonymous commenting is available, but is not intended to shield those taking pot shots at those of us challenging PBLA. If you are here to do that, please use your name.