Mathematics Exams: An Alternative Perspective

An idea for Maths Exams

We’ve had plenty change within Scottish education recently. In this blog I’m not advocating for more change, merely airing something I’ve been thinking about, as an idea for ‘next time’.

This is a thought experiment to start a conversation. I’d enjoy hearing what you make of the ideas I will discuss.

In this blog I’ll consider 3 questions:

1. What is mathematics?

2. What does a mathematics qualification suggest about a pupil’s competences?

3. What am I proposing we do for assessment?

What is mathematics?

There is no universally agreed definition of mathematics. Aristotle described the subject as ‘the science of quantity.’ This and other definitions which focus on magnitude and counting fail to account for much of the content subject. Many areas of mathematics, of course, bear no obvious relation to measurement or the physical world.

There are formalist definitions such as ‘Mathematics is the manipulation of the meaningless symbols of a first-order language according to explicit, syntactical rules’ (Snapper, 1979). However, formalist definitions seem to make the symbols and notation the object of study. These definitions ignore both the physical and mental meaning of mathematics.

Henri Poincaré moved beyond simplistic ideas of number and symbols stating that ‘Mathematics is the art of giving the same name to different things.’ This quote hints at one of the essential ideas in our subject: the act of generalisation. Taking the specific, spotting patterns and relationships and extracting an abstract generalisation that is independent of the specific. Karl Freidrick Gauss took the perspective that mathematics was about ideas, famously arguing that ‘What we need are notions, not notations’.

A universal definition of the subject is unlikely to be agreed upon. There is a tension which arises from the subject being viewed both as an art and science, which is perhaps responsible for this. In Scottish education mathematics is often labelled as numeracy and considered to be a life skill. In his famous piece “A mathematicians lament” Paul Lockhart (2002), takes a stance against this sort of utilitarian perspective.

‘It would be bad enough if the culture were merely ignorant of mathematics, but what is far worse is that people actually think they do know what math is about— and are apparently under the gross misconception that mathematics is somehow useful to society! This is already a huge difference between mathematics and the other arts. Mathematics is viewed by the culture as some sort of tool for science and technology. Everyone knows that poetry and music are for pure enjoyment and for uplifting and ennobling the human spirit (hence their virtual elimination from the public school curriculum) but no, math is important.’

I am of the opinion that somebody who studies the subject is learning:

· Procedural fluency

· Conceptual Understanding

· Mathematical Behaviours

The subject can be viewed as a domain with many facts, theorems, definitions, rules and procedures to master. This perhaps relates to what Skemp (1976) would call Instrumental Understanding. I refer to this as Procedural Fluency.

Skemp also discusses Relational Understanding which is about the ideas underpinning all of these instrumental understandings. It is about the network of connections and relationships within and between each of these understandings. It is about the principles that govern these relationships. I refer to this as Conceptual Understanding.

However, while Skemp’s ideas of Relational and Instrumental Understanding are necessary to capture what learning mathematics involves, I do not feel that they capture it fully. The third type of “understanding” is about the actions and thoughts of a mathematician. To paraphrase Mark McCourt “Being a mathematician is a way of existing in the universe.” I refer to this as Mathematical Behaviours.

Cuco, A., Goldenber, E. P., & Mark, J. (1995) lay out what they call Mathematical habits of mind. They suggest that pupils should be:

· Pattern sniffers

· Experimenters

· Describers

· Tinkerers

· Inventors

· Visualisers

· Conjecturers

· Guessers

These are the sort of behaviours of somebody who is in a state of ‘free play’ with some piece of mathematics. I am of the opinion that a mathematics education should result in pupils who have developed a range of mathematical behaviours, in addition to a robust Instrumental and Relational Understanding. The tasks we offer pupils shouldn’t be solely focused on the destination. The journey to completion and the actions that pupils take to get there is just as important. This goes beyond the models used, or the strategies invoked. It is about developing a language which captures the mathematical actions, such as those listed. It is about developing an awareness, in pupils, of these actions as being available. A pupil who is behaving mathematically will demonstrate these behaviours spontaneously, even when not explicitly asked to do so by the teacher or through some prompt in the task.

Anne Watson (2015) captures some mathematical behaviours in the table above. This is a useful point of reference. A question one might consider when teaching is: in addition to developing procedural fluency, and conceptual understanding how often do we plan to develop these specific mathematical behaviours?

Learning to use and apply simple mathematical procedures and ideas in everyday life is, and should be, a basic requirement of any education. For some, learning the subject is about setting the groundwork for later study in other fields such as physics, engineering or computer science. For these sorts of reasons, mathematics is a useful subject. But usefulness is not what drives or has driven most pure mathematicians through the ages. The enquiring, conjecturing minds pursuing pure thought for its own sake, has led to most of the great works in the subject’s history.

I offer the following conjecture: many of our pupils spend 12 years “studying” mathematics while rarely ever doing mathematics.

There would be some who argue through the lens of the knowledge versus skills dichotomy. I think this is academic. In stating Mathematical Behaviours separately from Procedures and Concepts I hope to draw attention to them and make them explicit.

What does a mathematics qualification suggest about a pupil’s competences?

I would hope that for a pupil achieving say, National 5, that they had a grasp of a certain level of content. I’d hope that a candidate who passes would have fluency with some fundamental processes and understanding of relationships underpinning those. We operate in a comprehensive system and, as such, at least a grade C pass *should* be within the grasp of the majority during their schooling.

One concern is that there is very little genuine problem solving in our examinations. The last question in the paper can generally be relied upon to be something previously unseen, but most questions are textbook questions with slightly different contexts. When the examiners do include genuine problem solving the pupils don’t score well. However, this makes a lot of sense as we shall now discuss. Going back a few years to when we had Credit level with separate Knowledge & Understanding and Reasoning grades, the threshold percentages for Reasoning were much lower. Also, over time, the Reasoning questions become more routine.

I suggest pupils don’t do well on non-routine problem solving in exams for two reasons:

1. Maturation, which I’ll discuss in the next section. Essentially it is hard to problem solve with mathematics which you’ve only recently learned.

2. That teaching is driven by past exams, which dictates that the focus is on the problems that routinely appear.

“The tasks presented by high stakes examinations and textbooks largely determine the types to ask that are used within classrooms.” Burkhardt & Swan (2013). It is often the case that being able to do questions which resemble past examination questions is viewed as the “goal state”. This can then lead to an emphasis, in teaching, on procedures at the expense of concepts, behaviours and problem solving. If the exam system does not value mathematical behaviours or conceptual understanding as highly as procedures (all of which essential in learning mathematics, as opposed to learning how to pass exams) then it is little wonder that they can be neglected.

As an aside, I suggest, perhaps controversially, that teaching pupils to pass exams is a real problem. The popular booklet Nix the Trix lists a whole range of “tricks” which pupils are sometimes taught without any understanding – purely with a focus on passing exams. While these tricks may help pupils pass the current level, they are seldom a sound basis for further study and often create problems for future learning. The inclusion of questions which test conceptual understanding or require problem solving rooted can encourage teachers to ensure pupils are mathematically proficient.

While, as individuals, we may not have the power to change assessments at a national level we can certainly influence the design of internal assessments in our own schools. I firmly believe in the mantra “if you teach the mathematics and build a mathematician, the exams take care of themselves”.

What do I propose for assessment?

I’m in favour of the retention of exams. I’ve yet to be convinced that coursework is meaningfully valid or reliable as an assessment model in mathematics. In the reality of busy schools, I think that there are serious consequences that would need to be carefully considered. Although not a coursework model, the assessment approach for National 4 hasn’t filled many teachers with confidence.

Recently, all of our national exams were increased in length. They are all rather long and were extended to take account of the removal of UASPs even though many/most schools still use them. Rather than assessing more content in the same manner, I’d have liked a restructuring.

In advocating a retention of exams, I’d suggest we have three sections:

1. Concept focused ‘multi-choice’ hinge questions

2. A range of questions of the same ilk as just now, getting increasingly challenging through the paper.

3. A ‘mathematical thinking’ section.

Concept Focused Multi Choice

Research seems to suggest that well written conceptual questions in exams can help us to increase focus on conceptual teaching. I like multiple choice as they are short and sharp and can be precise tools in ascertaining understanding. I wouldn’t want to have 20 questions like in the old Higher. Instead I’d rather have no more than 10 of them, which are focused precisely on concepts and relationships. I want well written questions where deep understanding is needed. In old Higher the multi-choice test included a lot of basic procedural work. I’m not advocating this.

Our exams are primarily focused on procedures, although a certain element of conceptual understanding is clearly required to answer many of the questions. It is hard to create assessment items which focus on concepts only, without invoking some element of procedural fluency. However, we’ve had questions recently which make a fair attempt at assessing concepts, including:

Cos60 = 0.5, what is Cos240?

Having the concept focused multi choice questions, I suggest, would hopefully positively influence the range of tasks pupils were exposed to in school. We need a succession of layered experiences, other than drill exercises, to develop this conceptual understanding. The tail does wag the dog. A key reason, in my experience, that people were uncomfortable with CfE was that there was no indication of the what the end point would be. The endpoint assessment, more than anything else dictates the curriculum. As such, let’s make the endpoint assessment different so that it makes pupil experience of the curriculum is richer. I suggest that this would be much more effective than spuriously including words such as “creativity” in the course documents.

A range of questions like just now

The exams we have just now are useful and include a good range of questions from basic application through to ‘working backwards’ and ‘application of two ideas’ questions. I would suggest that the bulk of the exam continues to be focused on this range of questions. There are certain mathematical ideas and processes we might expect pupils to have fluency with, for instance finding the equation of a straight line. In this section of the exam there would be one or two genuine, unseen, problem solving questions which require National 5 maths. These would likely be considered ‘A’ questions.

A Mathematical Thinking Section

Previously I suggested that maturation was an important reason for pupils performing poorly in problem solving. Essentially, problem solving using National 5 mathematics is hard for pupils who have just been learning National 5 maths. This mathematics hasn’t totally become assimilated into the pupil’s schema. It begins to make more sense with continued study of mathematics.

We often saw this idea of maturation come to light in Higher. Everyone used to teach straight line at the start of the year. Most pupils didn’t find this trivial. They would make mistakes, like using points which don’t lie on the line when finding equations of medians etc. However, come the end of the year almost every passing Higher candidate could do pretty much any straight-line question – even those that were non-standard. The reason is that the ideas in straight line had had a period of maturation. They might make calculation errors, but generally the pupils ideas of what to do would be correct.

On the contrary, the topics taught later in the course would be done badly if assessed in the exam in any sort of non-routine way. This would have been logs and exponentials, for instance. Nevertheless, six months later, during Advanced Higher those same pupils would routinely invoke the solving of exponential equations with relative ease in all sorts of situations. So maturation matters.

Colin Foster writing in MT (Feb 2019) captures the phenomena we see with the genuine National 5 problem solving questions.

“it is perfectly possible to have all of the necessary techniques safely inside your toolbox and yet not see how they could help you solve the problem you are tackling. The teacher feels frustrated, because they think that the students ought to be able to solve the problem. They apparently know everything they need to know, but they do not mobilise it in the particular situation they are presented with.”

This is something that I’m sure many teachers can relate to. He continues:

“One reason for this may be that the students have met the relevant content only in a narrow range of contexts and have not seen how it might be applied more widely. Another reason may simply be that they have encountered the relevant content too recently. When learning a language, students do not spontaneously and fluently use the vocabulary they have just learned. It needs time to bed in. Similarly, if we want students to make sophisticated use of what they know, it might be better to rely on mathematical content that was learned some time ago and is quite robustly known. Content learned 2 years previously is a rough rule sometimes used at the Shell Centre in Nottingham. … It also acknowledges that if the problem-solving demands are high, other demands, such as procedures and concepts, may need to be lower.”

With this in mind, if we value genuine problem solving and want to see pupils demonstrate this, then we might need to pose questions in this third section of the exam which are at a level below.

What might these questions look like? Well, the UKMT Junior questions might be somewhere to get inspiration from. They use maths which is wellestablished for National 5 pupils. However, they require pupils to reason with these ideas on a sophisticated level and apply problem solving strategies. The two questions below would be quite difficult for an S2 pupil, but provide just the right amount of challenge for a National 5 candidate to apply their previously learned material in a problem solving context.

While it might seem strange to include content which is not at the level being assessed, this is not a new idea. Many of the marks awarded at National 5 are not for National 5 skills. Further, in the 1990s Higher Maths papers frequently featured questions which were Advanced Higher level! Pupils would be given an example and then asked to complete a problem based upon their reading of the example.

As we know the pupils have the knowledge to do this, the focus for preparation for this part of the exam may focus around strategy. A framework such as the one laid out in Thinking Mathematically by Mason, Burton and Stacey might well be a useful point of reference.

The final question in the paper, would be something quite radical. Rather than being a problem to be solved, it would be a prompt for the pupil to explore. The marking of this might be based upon comparative judgement, where value is put on the evidence of mathematisation rather than on technical competency. What is valued here is not the “answer” but the pupils mathematical creativity, tinkering, conjecturing, visualising and experimenting.

Potential questions might be in the style of the ATM’s Points of Departure. Two examples are shown below.

What we value here is the mathematical behaviour. While these tasks lend themselves better to a longer protracted engagement, allowing say 20 or so minutes to ‘play’ with one prompt could still be valuable in allowing pupils to demonstrate their mathematical attributes beyond “what they know and understand”.

Conclusion

I’ve suggested that learning maths is about procedures, concepts and behaviours. Our exam system values procedures and, to a lesser extent, concepts. It places little stock in mathematical behaviours, despite much of the curriculum rhetoric being focused on these ‘softer’ skills.

The exam system does influence the enacted curriculum and the associated teaching focus. If our exams changed to reflect my idea, or some other idea, then the teaching and enacted curriculum would also change.

The idea of an exam I have laid out is far from perfect. I have no idea if it would work, but it tries to bridge the gap between a dry “regurgitate this” style exam to one where pupils have a realistic chance of showcasing their problem solving skills.

While I’ve focused on National 5, similar arguments could be made for Higher and Advanced Higher, where pupils problem solve and ‘tinker’ with ideas at the level below, for some of the marks in their exam.