Well the previous blog spoke about how standardized tests were all but accurate means of determining knowledge or ability. This blog shows the cheating and how easy it would be to do so and how to truly avoid it.
The previous blog showed how some of the questions in a standardized test did nothing but ask for trivia. So imagine that you are in film school and the question on one of your exams is to know the name of a particular widget on a camera that hasn't been produced since the 1980s. The student has never actually touched this camera and probably never will and this widget is called something different by another company on their camera if they even have it as an optional add-on. The class is supposed to be one where you learn how to take proper shots for film-making. Some students study for the test and focus on the nonsensical material and they get A+ on their test, some students have no interest in wasting their time in studying nonsense and pay for a cheat-sheet and also get the A+. We learn that cheaters exist and that they succeed more often than not, and we refuse to think about how it is even possible or how to properly avoid it.
It would be easy to spot by interviewing the student and seeing how neither one can have a genuine conversation about how to do a proper camera shot. The first one wasted their time in knowing unnecessary trivia about widgets and the second one wasted their money on a cheat sheet that informs them about these same widgets. Conclusion? Many of the A+ students out there don't have the basic understanding of the field they are supposed to be learning.
Easy experiment to do with a first year university student. Read the same textbook that the student is supposed to have read and have a conversation about what this textbook is saying. Don't ask specific questions because this will turn into trivia like questions and the student will end up quoting the appropriate passage. If you ask them what this passage means, they will be at a loss for words and probably attempt to repeat the quote.... maybe they will introduce their own words to replace the actual quote but when you ask them to explain to you what it actually means, they will be unable to do so. This is why cheating is so easy, the required answer is but a simple quote without having to explain what the quote means.
This will only work with first year students because part of the university education is to learn the specialized lingo or jargon of their field and know how and when to use it.... What is unfortunate is that since they never really understood what they were learning in the first year they tend to be encyclopedias of trivia without really understanding what the implications are. Then they use jargon to fake a knowledge that they do not truly have. Imagine when this A+ student becomes the next professor for the next generation of students and this type of quick testing does to genuine knowledge and understanding?
Well the world of employment has already noticed it, which is why they are more interested in the real world working experience of the newly graduated professional than his actual academic mark. Its quite easy to see how much this A+ graduate lacks the knowledge needed in the field he has studied because they can't get away with using trivia knowledge to prove their expertise and they definitely can't use jargon to fool the one who truly knows the field.
Consider George Orwell's example of an intellectual paragraph full of jargon:
"On the one side we have the free personality: by definition it is not neurotic, for it has neither conflict nor dream. Its desires, such as they are, are transparent, for they are what institutional approval keeps in the forefront of consciousness; another institutional pattern would alter their number and intensity; there is little in them that is natural, irreducible, or culturally dangerous. But on the other side, the social bond itself is nothing but the mutual reflection of these self-secure integrities. Recall the definition of love. Is not this the very picture of a small academic? Where is there a place in this hall of mirrors for either personality or fraternity?-----from an Essay on psychology in Politics (New York)"
Just what is that supposed to mean? Its nonsensical jargon that sounds intellectual but is not really saying anything of substance. Words are coming out just like sound comes out of a trumpet but put the words together they seem to sound sensible just like when you play individual notes you demonstrate that you can play those notes, but you aren't playing any music. Well universities tend to produce this type of graduate. He can quote trivia that is meaningless for actually solving a situation at hand and he speaks jargon about stuff that really doesn't make any sense (even to someone who knows the subject at hand) and is proud of having received his A+ for knowledge he doesn't truly have. Imagine someone who has reached the level of professorship and starts quoting jargon to the unsuspecting students. He is perceived as a brilliant professor by the administration which hired him (because they don't know anything about the subject either) and students who begin to learn the jargon believe this quack to be brilliant as well and they parrot his sayings without comprehension.
Sophistry is about the appearance of wisdom and the current methods of testing promote sophistry and encourage cheating. You don't have to know anything, you just have to fake that you know it long enough to jump whatever hurdle that immediately tests you. There is some statistic out there that claims that students only retain about 20% of what they learned in university, perhaps the truth of the matter is more likely that the student only truly learned 20% of what they should have?
I come back to my example of the widget on this non-existent camera as a test question to determine your knowledge of taking a proper shot or as a question on a standardized test for seeking employment. Is it any wonder that the university graduate or the newly hired employee is seen as not having any real-world experience or knowledge to do the basic camera operator? Now multiply this trend for every single area of employment..... Is it any wonder why economists could not prevent the last economic downturn and that even today, most are scratching their heads in trying to figure out how it happened in the first place?
If you are in a car and you put the car on neutral and the car is coasting down a hill, you congratulate yourself in making a car move without expending any gas (this is capitalism of the 19th and 20th century).... now suddenly, you notice that your car is heading for a shallow lake and that your feet are getting wet..... how do you solve it? You turn the car engine on and move forward faster? Oh the engine is flooded.... not to worry, our brilliant minds have found a way to start the car despite the rising water of the lake that we are still sinking/sliding into. Congratulations to the human ingenuity, but wait a second, the driver decides that he needs to move forward because he sees the shore on the opposite end of the lake. This seems sensible because every so called expert reason that moving forward is good, and seeing the shore proves that it is an attainable goal and they all use jargon to explain and convince the driver that his initial destination is correct. (the driver seems to have been given the authority to throw out of the car anyone he feels is a burden to his driving)...
The true thinker, who is not impressed with jargons, trivial knowledge of nonsense or self-important drivers suggests that perhaps putting the car in reverse may be the better solution because the shore behind is much closer, we don't know how much deeper the car may still sink and perhaps we could temporarily exit the vehicle to push and pull it out of its current predicament. The true thinker is thrown out of the vehicle for being so negative and not helping to solve the problem with the popular but wrong direction that the pseudo-intellectuals with A+ PhD degrees and the drivers all seem to agree upon.
And that is why it is so easy to cheat on tests. The test givers can't recognize the intelligence that the test is supposed to determine and the tests themselves or the required written projects themselves focus on trivialities and jargon, both of which are extremely easy to fake.
The A+ mark does not indicate intelligence or knowledge. Positions or titles do not determine genuine capability. When the lunatics have taken over the asylum, how does one find a true physician?
Tuesday, January 12, 2010
Wednesday, January 6, 2010
Standardized Tests
One of the most foolish ways to test one's knowledge or intelligence is through standardized tests. These tests have the underlying assumption that we all view things identically and ignore the fact that the inventor of the questions may himself have flawed or inaccurate information. Just remember how Einstein himself was considered a failure in grade school and his own peers in the realm of physics refused to publish him. Over the years some modifications have been introduced in the tests to make it less ridiculous.
When it comes to simple questions like "what is 2+2", the answers are not ambiguous. I have heard though, that even in advanced math, 1+1 is not always equal to 2 but I have not personally studied advanced math so I will continue with my grade school assumptions that 1+1=2. But I have studied other knowledge areas in university and some of these areas have ideas of thoughts that sometime contradict other ideas of thoughts and each position argue their points well. There is "proof" that one should punish a criminal because he is responsible for his actions; alternatively, there is also "proof" that one should rehabilitate a criminal because society has not educated this individual properly enough and that society must pay the price to re-educate the criminal. So it comes down to a matter of a "belief" system to determine which side of the equation that you will fall on.
The problem comes when someone creates a standardized test which asks how best to solve the criminal problem. Naturally, no one ever asks a question so directly in these tests, the questions are much more subtle, but if your "belief" system is against that of the one who created the questions and proper answers, then you failed those questions and you get a lower score on your permanent record. If you are applying for a prestigious career, these tests guarantee that everyone will think the same and be on the same page. Independent thought will not be allowed and when the ship sinks, no one on board will know what happened or why because everyone was blinded in the same way. Government bureaucracies would also function in a similar blindness because everyone in the corrections department would have been tested with the assumption that the best answer was "rehabilitation is best" while everyone in the law enforcement department would have been tested on the assumption that the best answer was "punish the criminal"..... This makes for an interesting schizophrenic bureaucratic department that the political leader must somehow manage especially if the political leader has a third viewpoint on how criminals should be treated.
In the early 20th century, IQ tests were performed on newly arrived immigrants to the US. It was discovered that the average score of intelligence for the immigrant was always 20 or so points lower than that of the average US citizen. Later they discovered that most of the questions were culture specific and that the newly arrived immigrant would have had no idea what the proper answer was. Imagine that the question is "What is the Big Apple" and the immigrant mentions something about a tree with a boost in special fertilizer..... WRONG says the one who corrects the answers (if the one who corrects these tests has no human judgment in determining what may be valid or not, then perhaps he should be in another field of employment).... Everyone knows that the "Big Apple" is a reference to New York and that is why the US citizen had higher marks in these IQ tests.
Today we have come a long way in avoiding such pitfalls and our tests are a bit more sophisticated and we allow less cultural references to be used as a way to test IQ. But the cultural reference mistakes have been replaced with ambiguous answers based upon what the tester believes to be correct when other answers are also debated in intellectual forums as correct. The one who takes the test must determine what the tester believes and then answer accordingly, this does not demonstrate intelligence or knowledge but rather the willingness to adapt to what the tester wants. And if these tests are only to determine who is willing to be a "yes-man", then the tests should be called as such.... The "Willingness to Submit to the Philosophical Idea of the Day" test. Naturally, the true thinker will be excluded from this exercise and everyone who passes this test will all run in the same direction without understanding why they are running nor where they are going..... and when the cliff approaches, most won't recognize it until they have run off of it while the leaders who led us off the cliff will cry out in glee that we are now flying high!
So when 2 or more answers on a multiple choice are equally valid dependent upon the school of thought you ascribe to, the only way to succeed is to understand the philosophy behind the tester, and while you figure that out and succeed in passing the test, you have not demonstrated superior intellect or superior knowledge, instead you have guessed the intention of the tester and acted like a proper puppet. The tests claim that they are looking for people with superior intellect and superior knowledge but they penalize them if they answer the test truthfully, instead they are really looking for clones who will all think and speak alike in a hive mind.
Questions I have seen in some tests that either test intelligence or knowledge in a specified field of study.... (note, these are metaphorical approximations with exaggerated foolishness to make a point because the questions are more subtle and the reader would have to have specialized knowledge in a very specific field to see the flaw in the question and that the average person would probably not see the problem).....
Q1-Who won the Stanley Cup in 1983? (Nice knowledge of trivia but how is it relevant for determining knowledge of sociology or determining that a prospective student has sufficient IQ to be accepted at university?)
Q2-Finish the series 1, 2, 3.... Choice of answers include A)4 B)5 C)6 D)10.....
you choose A) because each number increases by one integer.
you choose B) because each succeeding number is the sum of the previous 2 numbers.... 1+2=3, 2+3=5.
you choose C) because each succeeding number is the sum of all previous numbers put together.... 1+2=3, 1+2+3=6.
you choose D) because this a number system that only has 4 digits and is not a decimal based system..... 1, 2, 3, 10, 11, 12, 13, 20, 21, 22, 23, 30, 31, 32, 33, 100 (and these were numbers from 1 to 16)
Q3-Finish the series 1, 2, 3, 5.... Choice of answers include A)7 B)8 C)14 D)13...
you choose A) because these are prime numbers in sequence.
you choose B) because you add the sum of the previous two numbers....
you choose C) because you multiply the previous 2 numbers and subtract the first digit in the sequence.
you choose D) because you multiply the previous 2 numbers and subtract the digit which is located 2 positions away from the answer.
Q4-What is the widget found underneath an outdated camera that is no longer manufactured and no one will ever use again? (nice piece of trivia, but how does knowing this trivia determine my knowledge of how to take a great camera shot?)
Q5-When was gunpowder first used? (in a European context or Chinese?)
Q6-What year was the new World discovered? (considering the Viking explorers who did not record their time accurately? or the politically recognized official answer?..... not to mention the theory of the Bering Strait crossover from Asia?)
Q7-Who used the first computer? (did the test giver read the latest archaeological journal of the discovery of a mechanical computer found on a sunken Roman ship or is the test giver merely using last year's information?)
For a fictional example of standardized tests and how it won't recognize someone truly brilliant but only find the above average within a pool of average people, I recommend the book: "Cryptonomicon" by Neal Stephenson. The protagonist does a standardized test which he only answers one question in a period of 2 hours and gets it "wrong".... and yet, had someone with proper judgment on interpreting results of tests been correcting the test, he would have immediately seen the brilliant nature of the protagonist....
So the conclusion and the point of this rant against standardized tests is that you won't find the true thinker with this tool and you will only get identical thinking clones who don't know how to think outside of the box.
When it comes to simple questions like "what is 2+2", the answers are not ambiguous. I have heard though, that even in advanced math, 1+1 is not always equal to 2 but I have not personally studied advanced math so I will continue with my grade school assumptions that 1+1=2. But I have studied other knowledge areas in university and some of these areas have ideas of thoughts that sometime contradict other ideas of thoughts and each position argue their points well. There is "proof" that one should punish a criminal because he is responsible for his actions; alternatively, there is also "proof" that one should rehabilitate a criminal because society has not educated this individual properly enough and that society must pay the price to re-educate the criminal. So it comes down to a matter of a "belief" system to determine which side of the equation that you will fall on.
The problem comes when someone creates a standardized test which asks how best to solve the criminal problem. Naturally, no one ever asks a question so directly in these tests, the questions are much more subtle, but if your "belief" system is against that of the one who created the questions and proper answers, then you failed those questions and you get a lower score on your permanent record. If you are applying for a prestigious career, these tests guarantee that everyone will think the same and be on the same page. Independent thought will not be allowed and when the ship sinks, no one on board will know what happened or why because everyone was blinded in the same way. Government bureaucracies would also function in a similar blindness because everyone in the corrections department would have been tested with the assumption that the best answer was "rehabilitation is best" while everyone in the law enforcement department would have been tested on the assumption that the best answer was "punish the criminal"..... This makes for an interesting schizophrenic bureaucratic department that the political leader must somehow manage especially if the political leader has a third viewpoint on how criminals should be treated.
In the early 20th century, IQ tests were performed on newly arrived immigrants to the US. It was discovered that the average score of intelligence for the immigrant was always 20 or so points lower than that of the average US citizen. Later they discovered that most of the questions were culture specific and that the newly arrived immigrant would have had no idea what the proper answer was. Imagine that the question is "What is the Big Apple" and the immigrant mentions something about a tree with a boost in special fertilizer..... WRONG says the one who corrects the answers (if the one who corrects these tests has no human judgment in determining what may be valid or not, then perhaps he should be in another field of employment).... Everyone knows that the "Big Apple" is a reference to New York and that is why the US citizen had higher marks in these IQ tests.
Today we have come a long way in avoiding such pitfalls and our tests are a bit more sophisticated and we allow less cultural references to be used as a way to test IQ. But the cultural reference mistakes have been replaced with ambiguous answers based upon what the tester believes to be correct when other answers are also debated in intellectual forums as correct. The one who takes the test must determine what the tester believes and then answer accordingly, this does not demonstrate intelligence or knowledge but rather the willingness to adapt to what the tester wants. And if these tests are only to determine who is willing to be a "yes-man", then the tests should be called as such.... The "Willingness to Submit to the Philosophical Idea of the Day" test. Naturally, the true thinker will be excluded from this exercise and everyone who passes this test will all run in the same direction without understanding why they are running nor where they are going..... and when the cliff approaches, most won't recognize it until they have run off of it while the leaders who led us off the cliff will cry out in glee that we are now flying high!
So when 2 or more answers on a multiple choice are equally valid dependent upon the school of thought you ascribe to, the only way to succeed is to understand the philosophy behind the tester, and while you figure that out and succeed in passing the test, you have not demonstrated superior intellect or superior knowledge, instead you have guessed the intention of the tester and acted like a proper puppet. The tests claim that they are looking for people with superior intellect and superior knowledge but they penalize them if they answer the test truthfully, instead they are really looking for clones who will all think and speak alike in a hive mind.
Questions I have seen in some tests that either test intelligence or knowledge in a specified field of study.... (note, these are metaphorical approximations with exaggerated foolishness to make a point because the questions are more subtle and the reader would have to have specialized knowledge in a very specific field to see the flaw in the question and that the average person would probably not see the problem).....
Q1-Who won the Stanley Cup in 1983? (Nice knowledge of trivia but how is it relevant for determining knowledge of sociology or determining that a prospective student has sufficient IQ to be accepted at university?)
Q2-Finish the series 1, 2, 3.... Choice of answers include A)4 B)5 C)6 D)10.....
you choose A) because each number increases by one integer.
you choose B) because each succeeding number is the sum of the previous 2 numbers.... 1+2=3, 2+3=5.
you choose C) because each succeeding number is the sum of all previous numbers put together.... 1+2=3, 1+2+3=6.
you choose D) because this a number system that only has 4 digits and is not a decimal based system..... 1, 2, 3, 10, 11, 12, 13, 20, 21, 22, 23, 30, 31, 32, 33, 100 (and these were numbers from 1 to 16)
Q3-Finish the series 1, 2, 3, 5.... Choice of answers include A)7 B)8 C)14 D)13...
you choose A) because these are prime numbers in sequence.
you choose B) because you add the sum of the previous two numbers....
you choose C) because you multiply the previous 2 numbers and subtract the first digit in the sequence.
you choose D) because you multiply the previous 2 numbers and subtract the digit which is located 2 positions away from the answer.
Q4-What is the widget found underneath an outdated camera that is no longer manufactured and no one will ever use again? (nice piece of trivia, but how does knowing this trivia determine my knowledge of how to take a great camera shot?)
Q5-When was gunpowder first used? (in a European context or Chinese?)
Q6-What year was the new World discovered? (considering the Viking explorers who did not record their time accurately? or the politically recognized official answer?..... not to mention the theory of the Bering Strait crossover from Asia?)
Q7-Who used the first computer? (did the test giver read the latest archaeological journal of the discovery of a mechanical computer found on a sunken Roman ship or is the test giver merely using last year's information?)
For a fictional example of standardized tests and how it won't recognize someone truly brilliant but only find the above average within a pool of average people, I recommend the book: "Cryptonomicon" by Neal Stephenson. The protagonist does a standardized test which he only answers one question in a period of 2 hours and gets it "wrong".... and yet, had someone with proper judgment on interpreting results of tests been correcting the test, he would have immediately seen the brilliant nature of the protagonist....
So the conclusion and the point of this rant against standardized tests is that you won't find the true thinker with this tool and you will only get identical thinking clones who don't know how to think outside of the box.
Subscribe to:
Posts (Atom)