IPB

Welcome Guest ( Log In | Register )


AZScoring
Posted on: Mar 11 2015, 09:45 PM


Member
**

Group: Coach Class
Posts: 13
Joined: 8-February 10
From: Tucson, AZ
Member No.: 455


QUOTE (JSC @ Mar 11 2015, 12:37 PM) *
In my mind, the problem with essay scoring is not that it's "subjective", because I believe that speech and interviewing scoring gets it right a very large percentage of the time. (And that it is not near as subjective as is popularly believed.) The problem is that the rubric sucks.



In AZ We've used both a very detailed, thought out rubric and the USAD rubric. Both present their own issues and benefits, in and out of the scoring room. But this discussion is about how they are scored not why they are scored. cool.gif
  Forum: Academic Decathlon C&S · Post Preview: #90865 · Replies: 13 · Views: 6,601

AZScoring
Posted on: Mar 8 2015, 08:26 PM


Member
**

Group: Coach Class
Posts: 13
Joined: 8-February 10
From: Tucson, AZ
Member No.: 455


QUOTE (The Evil Dr. Calculus @ Mar 8 2015, 11:58 AM) *
I just texted Jason, the Arizona State Scoring Chair. I'm sure he knows the answer.


Hopefully I'll address the questions/issues asked here...If not please point out what I missed.
That being said - the scoring (as far as the numbers) is black and white. If essay scores are >200 points apart it requires another read until any given set of scores are 200 or less apart.
The subject of quality of judges, training, and prompts is a whole other discussion.


On the first question.
I have a question about essay scoring at Texas State that some more experienced person may be able to answer. One of our varsities ended with a final essay score of 785. My coach tells me that the first two graders gave him polarized scores of ~980 and ~620. They naturally brought in a third scorer, who gave him ~820. However, this then prompted them to bring in a fourth judge, whose score I do not know. Is this supposed to happen, and what's the scoring technique when it does?

If those were truly the scores ~980 and ~620. Yes that requires a third read. and if the 3rd read was ~820. The essay score should have been averaged from the ~980 and ~820. Giving that particular student an essay score of ~900. I'm not sure why they would have done a 4th read. (again this is solely based on the scores reported)
Those 2 scores are within 200 points of each other.


Second question
This method strikes me as troublesome because of the case of a theoretical Decathlete who is scored 600 and 990 by two judges. A third judge is called. If this judge gives an 800, the student's score becomes 895; if the judge gives a 790, the score becomes a 695. As extreme as this sounds, it could happen, and it seems very arbitrary that a 10-point difference from one judge causes a 200-point difference in actual score.

Yep - it's math. This is why it's called a subjective test and we require more than one person(judge) to be involved with the score. Again - The subject of quality of judges, training, and prompts is a whole other discussion.

Third question
What happens if, say, the first two judges give 600 and 900, and the third gives 750, which is within 200 of both scores? Are the three scores averaged?

Well, that's an interesting question. I have 2 answers.

1)In the very particular and rare case of an exact split from the 3rd judge. The tie should go to the higher of the 2 scores. I always give the benefit to the student.

2) The unfortunate answer for the scoring software. Please keep in mind that this situation, while possible, is very unlikely. I can't remember when this has ever happened or come close to happening. The odds make it fairly unlikely.

The software will give a different answer based on judge order entry.

ex judge 1 900
judge 2 600
judge 3 750

the essay score is 675

ex judge 1 600
judge 2 900
judge 3 750

the essay score is 825

FYI - if it's not an exact split - the score will be figured from the 2 that are closest. ie ~600, ~900, ~751 - it will take the 751 and 900 scores.
  Forum: Academic Decathlon C&S · Post Preview: #90856 · Replies: 13 · Views: 6,601

AZScoring
Posted on: Feb 25 2015, 09:25 PM


Member
**

Group: Coach Class
Posts: 13
Joined: 8-February 10
From: Tucson, AZ
Member No.: 455


QUOTE (The Evil Dr. Calculus @ Feb 8 2015, 06:37 PM) *
I expect them some time in the next 48 hours. Jason and his wife are pretty amazing. This year, they had some problems in Arizona Region I (the northern part of the state), so he went up to Payson and scored Region I, while his wife stayed in Tucson and scored Region III. Everything went off perfectly.





thanks for the kind words!

Jason



QUOTE (madcap @ Feb 9 2015, 05:56 PM) *
QUOTE (The Evil Dr. Calculus @ Feb 8 2015, 08:37 PM) *
I expect them some time in the next 48 hours. Jason and his wife are pretty amazing. This year, they had some problems in Arizona Region I (the northern part of the state), so he went up to Payson and scored Region I, while his wife stayed in Tucson and scored Region III. Everything went off perfectly.
That's outstanding. I'm beyond thankful for people like them.





...and thank you for the kind words. I enjoy giving back to a program that meant so much to me in high school. Thank goodness I have a wife that 's willing to be dragged along.




  Forum: Academic Decathlon C&S · Post Preview: #90720 · Replies: 33 · Views: 20,871

AZScoring
Posted on: Mar 4 2014, 05:06 PM


Member
**

Group: Coach Class
Posts: 13
Joined: 8-February 10
From: Tucson, AZ
Member No.: 455


QUOTE (The Evil Dr. Calculus @ Feb 26 2014, 10:37 AM) *
Oceanography was my favorite super quiz topic of all time, and was from 2002-03.


Science Fiction was mine. Although I'm guessing most people on this board don't remember that one. geek.gif
  Forum: Academic Decathlon · Post Preview: #88276 · Replies: 261 · Views: 113,663

AZScoring
Posted on: Feb 7 2011, 11:52 AM


Member
**

Group: Coach Class
Posts: 13
Joined: 8-February 10
From: Tucson, AZ
Member No.: 455


QUOTE (Titoxd @ Feb 5 2011, 07:25 PM) *
Coaches, please double-check all of your Round 2 scores for irregularities. I checked both Mountain Pointe's and Marcos de Niza's scores and both schools had problems with Essay and Super Quiz scores. (You know the score is wrong when it is negative rolleyes.gif ) Some students have missing Super Quiz relay or written scores, since they got scores that are numerically impossible. The problem might be even more widespread, so please check all your result sheets and try to see if anything looks suspicious, and make sure people at AAD know about it.


Titoxd,

This is Jason from state scoring. Can I assume you are the coach from Marcos? You should have my email address already but if you don't please get it from Mr. Yetman or check with any other coach (sorry I don't want to post it here). I'd like to address your questions personally with specifics but for the time being I can at least comment in general here.

Both the essay scores indeed need another read. I'm not sure why it was missed but it unfortunately it was. In the case of Marcos it actually needs a 4th read.
As far "missing" relay scores no one from Marcos is missing a score. From the entire competition there were only 9 students that received zeros otherwise everyone else has a score. Those 9 students did not answer any questions correctly so they did not receive a score or rather received a score of zero. I'm not sure if that's what you meant by missing scores.

Aa far as getting a score that was numerically impossible. I was asked how it was possible that a student at Marcos received a 195 for super quiz (a numeric impossibility)? Looking at the PDF that was sent and reviewing the scores from the program, I see a score of 185 not a 195. And Ii know there was some question about how many answers that student got correct at the super quiz. According to what was scanned that student only answered one question correctly. It's possible the proctor made a mistake. If you email me we can discuss what the student answered vs what the correct answers were.

We are going to look at the super quiz sheet for the student in question and take care of the missing reads for the essay. If there are any other questions please let me know.

Jason
  Forum: Academic Decathlon C&S · Post Preview: #58128 · Replies: 30 · Views: 12,343

AZScoring
Posted on: Feb 11 2010, 08:32 PM


Member
**

Group: Coach Class
Posts: 13
Joined: 8-February 10
From: Tucson, AZ
Member No.: 455


yep her score changed because of the question that was disputed. While it did not affect the teams relay score her official score was changed/updated to reflect the answer change.
[/quote]
Well that explains it then. So you lose all the points if the question is disputed? That seems weird. Do we know what relay one it was specifically? I don't remember second guessing any of those last ones (but at that point we were all just excited to be done wink.gif ).
[/quote]

You don't automatically lose your points. But if a question is successfully disputed and the answer changes then the new answer is what counts in the final official results. In this case she answered [SOMETHING] but the correct answer (after review) was determined to be [SOMETHING ELSE]. So yes it's a bummer that ultimately she didn't (as well as others) get points for that question but she did answer it wrong (after the dispute was settled). There are other people on the flip side that got points (in the official tally) that did not receive them at the relay. It certainly goes both ways. Just a bit of info as well. The relay scores are never the "official" scores...for this very reason.

Sometimes we'll set questions to have multiple correct answers but for this one it was a matter of the better answer was different than what USAD originally listed as the correct answer. The decision affected everyone equally so she wasn't penalized or rewarded more than any other person. I can't post the specific question with answers but It was [A QUESTION].
  Forum: Academic Decathlon C&S · Post Preview: #36020 · Replies: 97 · Views: 33,255

AZScoring
Posted on: Feb 11 2010, 04:34 PM


Member
**

Group: Coach Class
Posts: 13
Joined: 8-February 10
From: Tucson, AZ
Member No.: 455


QUOTE (j_ernzen @ Feb 9 2010, 05:08 PM) *
QUOTE (madcap @ Feb 9 2010, 01:55 PM) *
Thank you! You seem to have John Ernzen's score wrong, though.

By the way, this isn't confirmed or anything but we think there may be a problem with Alice's score. Because in SQ she got all 5 for the relay but then her score sheet says she had a 905, which is impossible with only missing written questions. What we think may have happened (again, all of this is just our own speculation) is that they only counted 4 of the relay questions which would give her a 985 in actuality (which makes more sense anyway, considering how well Alice knows the SQ). Again, I'm not sure if that's what happened but some mistake must have been made somewhere along the line.



yep her score changed because of the question that was disputed. While it did not affect the teams relay score her official score was changed/updated to reflect the answer change.
  Forum: Academic Decathlon C&S · Post Preview: #36002 · Replies: 97 · Views: 33,255

AZScoring
Posted on: Feb 9 2010, 09:54 PM


Member
**

Group: Coach Class
Posts: 13
Joined: 8-February 10
From: Tucson, AZ
Member No.: 455


QUOTE (madcap @ Feb 9 2010, 02:25 PM) *
Just one more thing (I hope): someone on this board reported that Brendan McNicholas of Border creek scored 7,819.7 as a honor... is that correct? Maybe he's also 1000 points off?



thanks madcap. I forgot the scoring software doesn't play nice when exporting perfect scores. wink.gif Everything should be correct now. Please anyone give me a heads up if they see something off.

Madcap could you maybe edit out the quote of the old scores on your post so no one accidently quotes incorrect scores?

thanks!
  Forum: Academic Decathlon C&S · Post Preview: #35743 · Replies: 97 · Views: 33,255

AZScoring
Posted on: Feb 9 2010, 09:22 PM


Member
**

Group: Coach Class
Posts: 13
Joined: 8-February 10
From: Tucson, AZ
Member No.: 455


[/quote]

Thank you! You seem to have John Ernzen's score wrong, though.

[/quote]


Yes I did. But it's now corrected. Thanks for letting me know.
  Forum: Academic Decathlon C&S · Post Preview: #35739 · Replies: 97 · Views: 33,255

AZScoring
Posted on: Feb 9 2010, 08:27 PM


Member
**

Group: Coach Class
Posts: 13
Joined: 8-February 10
From: Tucson, AZ
Member No.: 455


QUOTE (PrussianBlue @ Feb 8 2010, 12:57 PM) *
And their scores, if possible.


# Region School Student Overall

1 - Honors

1301 IV Mesa Mountain View John Ernzen 8951.9
1303 IV Mesa Mountain View Alice Lin 8221.2
1702 III CDO Nicolas Baird 8127.0
1701 III CDO Anna Hainsworth 8055.2
0901 IV Gilbert Anna Borja 7993.1
1902 IV Hamilton Shelley Xu 7875.4
1401 II Boulder Creek HS Brendon McNicholas 7819.7
1703 III CDO Seungwoo Sun 7726.7
0901 III Buena Quentin Gunn 7562.7
1302 IV Mesa Mountain View Jayme Gosney 7472.9
1201 IV Dobson Betty Hsu 7412.6
1503 III Marana Mountain View Danielle Oxnam 7405.8
1903 IV Hamilton Stephen Bergauer 7372.7
1901 IV Hamilton Shaina Hasan 7362.6
0903 II Sandra Day OConnor HS Katherine Fackrell 7225.7
1602 IV Mountain Pointe Swaroon Sridhar 7222.9
2203 II North Canyon High School Shane Cummerford 7206.4
2003 II Deer Valley High School Tania Lebratti 7133.3
1403 IV Highland Nicole Kang 7117.1
0801 IV Mesa Matt Betz 7046.0
1801 IV Skyline Alex Weldon 7034.6
1503 IV Mesquite Erika Handly 7025.0
1502 IV Mesquite China Wang 7023.3
2002 IV Red Mountain Cory Hernandez 6943.6
1402 IV Highland Joe Carpenter 6921.4

2 - Scholastic

1306 IV Mesa Mountain View Tyler Searle 7874.7
1305 IV Mesa Mountain View Christine Adams 7700.2
1704 III CDO Jordan Neerhof 7148.6
1705 III CDO Ellie Strasser 7125.4
1406 IV Highland Humza Salim 7072.7
2004 IV Red Mountain Brianna Pantilione 7039.7
1304 IV Mesa Mountain View Joseph Horen 6970.9
1805 IV Skyline Jared Brooks 6771.3
1105 IV Westwood Nick Nelson 6749.4
1506 III Marana Mountain View Kate Edrington 6688.7
0905 III Buena Brian Sayler 6645.3
1604 IV Mountain Pointe Cailen McCloskey 6630.2
1904 IV Hamilton Kevin Marquez 6610.8
1804 IV Skyline Kevin Alarcon 6580.0
1106 IV Westwood Matthew Palmer 6543.6
1706 III CDO David Argentati 6529.7
1506 II Cesar Chavez HS Jennifer Madruga 6501.0
1905 IV Hamilton Beverley Makhubele 6499.6
1405 IV Highland Megan Riley 6492.0
2005 IV Red Mountain Kaci Jensen 6457.1
1204 IV Dobson Pablo Pena 6435.3
1304 II South Mountain HS Krista Coates 6373.2
2006 IV Red Mountain Jon Woodmansee 6322.7
1204 I St. Johns Anthony Holt 6316.7


3 - Varsity

1207 IV Dobson Joe Yu 8609.7
1309 IV Mesa Mountain View Alan Gieske 7740.6
1707 III CDO Rush Moore 7688.3
1709 III CDO Jordan Kurker-Mraz 7636.8
1708 III CDO Chelsea Parraga 7378.3
1109 IV Westwood William Daigle 7315.4
2007 IV Red Mountain Ian Mathews 6862.1
1307 IV Mesa Mountain View Zach St. Martin 6816.6
0709 IV Marcos de Niza Fahad Alam 6787.1
0608 I Kingman Academy Trenton Pinson 6542.5
1508 IV Mesquite Shannon McCurdy 6366.3
1509 II Cesar Chavez HS Khari Hunt 6336.6
1507 II Cesar Chavez HS Gai Le 6318.6
0807 IV Mesa Kevin Campos 6174.7
1508 II Cesar Chavez HS Alex Eakins 6159.7
1207 I St. Johns John Blevins 6130.3
0609 I Kingman Academy Jenna Lewis 6094.9
1308 IV Mesa Mountain View Jake Layton 5976.7
1509 III Marana Mountain View Rachel Laprevote 5947.3
1409 IV Highland Ryan Carr 5915.7
1908 IV Hamilton Bethany Grater 5813.0
0209 IV Valley Christian Nathan LeDeaux 5809.3
1108 IV Westwood Jacob Washburn 5780.8
0607 I Kingman Academy Ben Brantley 5776.2
  Forum: Academic Decathlon C&S · Post Preview: #35736 · Replies: 97 · Views: 33,255

AZScoring
Posted on: Feb 8 2010, 05:44 PM


Member
**

Group: Coach Class
Posts: 13
Joined: 8-February 10
From: Tucson, AZ
Member No.: 455


To any Region II coaches that did not receive individual scores sheets.

It was an oversight and by no means intentional. I sent acadecker (he emailed me and asked) and your regional coordinator copies of all the individual score sheets. If you have not received copies of the scores and want them soon, you can bug either of them; otherwise, sometime this evening I'll be sending out links to download PDFs of all the scores from across the state.

  Forum: Academic Decathlon C&S · Post Preview: #35611 · Replies: 20 · Views: 8,677

AZScoring
Posted on: Feb 8 2010, 05:11 PM


Member
**

Group: Coach Class
Posts: 13
Joined: 8-February 10
From: Tucson, AZ
Member No.: 455


Thanks for the kind words Dr.
Score sheets in RII were resolved. smile.gif someone figure out a way to clone myself and I'll score all 4 regions.
  Forum: Academic Decathlon C&S · Post Preview: #35598 · Replies: 20 · Views: 8,677


New Posts  New Replies
No New Posts  No New Replies
Hot topic  Hot Topic (New)
No new  Hot Topic (No New)
Poll  Poll (New)
No new votes  Poll (No New)
Closed  Locked Topic
Moved  Moved Topic
 

Lo-Fi Version Time is now: 22nd January 2018 - 11:40 PM