24 January 2023
Score 100% in your A-level NEA project, from 1 page of code!
So the headline has caught your attention. Good. I promise to deliver on it. But first, a little background.
When I taught A-level Computer Science, I always enjoyed the NEA projects. Some of my students struggled with the programming; more struggled with writing the report; but most enjoyed it to some degree. In several respects I believe it to be the most important part of the whole A-level in terms of learning. I often thought it would be instructive to have a go at such a project myself. I am sure many teachers would like the opportunity to do this, but most simply don’t have the time.
Last summer I was introduced to Wordle (https://www.nytimes.com/games/wordle), which I much enjoy playing. But like any good computer scientist, I hadn’t been playing long before I started thinking about how to write an automated solver. And then I thought: I should conduct this as a hypothetical A-level project! It’s certainly not a trivial challenge, but nor is it beyond the reach of an A-level pupil aiming for an A grade.
My finished Solver performs far better than I had hoped for. It is guaranteed to solve any official Wordle puzzle in no more than five attempts (six being the maximum allowed) and takes five for only 3.6% of puzzles – otherwise four or fewer. I know this because, as my report explains, I tested the finished solver against all 2,309 possible target words that the official Wordle puzzle may set.
From the outset I decided to set myself a voluntary constraint: to write the solver using a ‘functional programming’ (FP) approach – though in a mainstream programming language. I chose C# - but don’t switch off if that’s not your preference, because I have since translated the finished Solver into both Python and VB, to make it accessible to more teachers and students.
Adopting an FP approach requires more discipline than conventional programming, but it also offers many potential benefits including better testability, provability, and performance (through parallelisation) – I explain these within the report.
When you look at my code, you might initially think that you (and/or your students) ‘couldn’t possibly write that!’ Over the coming months, I want to persuade you otherwise: that you can learn to code this way, and you can teach it. I believe that – at A-level – we should teaching students to write code this way from the outset. Currently, it looks hard because it is unfamiliar – to those who have only known procedural and/or object-oriented programming. I understand this – I didn’t learn FP properly until my late ‘50s, having built the previous 20 years of my career on a very pure approach to the OOP paradigm. Undergoing a ‘paradigm shift’ doesn’t get any easier as you get older. Even Thomas Kuhn (who coined the phrase ‘paradigm shift’) said that true paradigm shifts take a generation – in other words they to wait for most of the old guard to die off! I can cope with my children thinking I am a dinosaur in certain of my views on the world, but I don’t want to be seen that way professionally.
I believe not only that FP is the future of programming, but that if we taught pupils this approach from the outset they would gain a stronger understanding of programming, be more productive, and make fewer mistakes. (Some of the most common errors in coding – for example ‘off by one errors’ – are much rarer in FP because there are far fewer opportunities to make them.)
You’ll be hearing much more from me on this theme in future blog posts. I am investing an enormous effort in the development of brand new resources, and a new custom-built online Integrated Learning Environment – ‘MetalUp.Express’ – designed specifically to teach teachers, and their A-level students, to code this way. If you are interested in gaining early sight of this work, please email me – rpawson@metalup.org).
Back to the NEA project … another benefit of FP is that the resulting code is also typically very terse. My complete C# code solution for an automated Wordle Solver is printed, quite legibly, on a single page of the report.The only thing not included in ths page are the constant data definitions – the two long lists of acceptable answers and guess words. (BTW, the Python translation is a similar length to the C#; the VB translation is slightly longer, not because of any deficiency in that language, but merely because of the syntax and conventions. I could have forced even the VB version to fit on one page by placing more than one statement on a line; this is technically permissible, but it is not common practice).
Being short does not necessarily make it very quick to write. I think I devoted an appropriate number of hours to the coding for an A-level project. Interestingly, as I describe in the report, the part that I thought would be easy took me weeks to work out (even though the resulting solution looks so simple you might wonder why I didn’t see it from the outset.) And the part I thought would be really hard turned out to be quite straightforward. That’s often the way, isn’t it?
Now in theory the A-level boards don’t view the quantity of code as an indicator of quality. But does the practice match the rhetoric? More than one teacher, with whom I had informally discussed my project, expressed doubt that I would be able to get a high score from just one page of code, no matter how clever it was. So I decided to put this to the test.
I wrote up the project in a report consistent with an AQA A-level NEA project submission. I chose AQA not only because I prefer that syllabus (having taught OCR for one year before switching), but also because I thought this project stood a better chance with AQA, for two reasons. First, because in the AQA mark scheme a higher percentage of the marks is based on the technical solution, and secondly because the AQA specification includes at least some Functional Programming, so AQA is at least aware of its importance.
Having completed the report, I then decided to mark it as against the AQA mark scheme (having been through the full AQA Teacher Standardisation training and having marked my own students’ projects in the past). For this project I awarded myself 100% (75/75 marks)! As Mandy Rice-Davies famously said, ‘Well he would, wouldn’t he?’
However, I then forwarded the report to an experienced AQA moderator and asked for an informal opinion. The moderator responded by saying that they thought it is was an excellent project, and that – had it been a real project, and had it been marked at 100% by the teacher - they thought the mark would have held. (To be clear, this does not imply they would have necessarily have given it 75/75, in fact no specific mark was offered. It means that they mark they would have given would have been withn tolerance, such that the original teacher mark would hold.) I consider this response to justify my claim of having an existence proof that you can score 100% from a single page of code. (I did also approach AQA about getting an ‘official’ mark. They showed considerable interest in my project, but regretted that they were not permitted to give an official mark because that would constitute an endorsement, for which there are very strict rules and constraints).
A couple of caveats:
- Less is only more if it is also better. This project got a full score for technical solution only because the coding technique is sophisticated and the resulting functionality is rich. Submitting one page of procedural code would be very unlikely to score well, IMO. Writing terse, functionally rich code, is not easier than writing pages and pages of conventional prodecural code, but it is much more satisfying, and – I suggest – makes it easier for the marker to justify a high score for Technical Skills. And that’s apart from the other well-known benefits of FP described earlier.
- To get a full score you have to do a very good job on the report, not just the technical solution. (You can judge that for youself when you read the report, but I’ll say more on this topic in future postings.)
Meantime, I encourage you to:
- download my full report (plus, optionally, the main code translated into Python, and into VB) from my website: http://metalup.org/wordlesolver/Description.html
- watch the video of the Solver taking on a live Wordle puzzle: https://www.loom.com/share/75b11691603d485fb7b2a356b4e51e94
- download the complete source code for the Wordle Solver, in C#, VB, or Python , with instructions on how to run it from scratch. You can clone the repository or just download a .zip of the complete solution: https://github.com/MetalUp/WordleSolver
This is the first of several blog posts that I intend to publish in relation to this project. In the next one I will list lessons that I learned from undertaking an A-level NEA project for myself: nothing to do with the use of FP techniques. Some have to do with the conduct of the project; many more have to do with writing the report.
Discussion
Please login to post a comment
I have fixed this. Hopefully the it should work now. Let me know and I will have another go.
Sorry for the hassle
This is a footnote to the project, strictly of interest to tech-heads, I suggest.
In the video I state that the Wordle Solver is deterministic, so that it will always come up with RAISE as the first (optimal) guess, and then, for the same outcome (marks) will always come up with the same second guess and so on.
I was quite shocked last week to discover that this is not true! I re-ran the same puzzle that I used in the video and instead of BLUDY as the second guess it came up with DUPLY. I have also heard from someone else that they ran my Wordle Solver and it came up with ARISE as the first guess.
A little investigation reveals that RAISE and ARISE get an equal score from the algorithm. They are both possible answers, and both leave a ‘worst case’ scenario of 167 remaining possible answers. (Actually, running the full simulation it turns out that RAISE is fractionally better overall, but - as I discuss in the report’s Evaluation section - the minimax algorithm currently calculates only to one level deep). The same is true of BLUDY and DUPLY as the second guess after a ‘5 greys’ outcome on RAISE.
When faced with equal-scoring next attempt words, the one picked by the solver is dependent upon the order in which they are presented. It needn’t be - that was just the simplest algorithm. But if the order in which they are presented is determined by the constant data lists, how can the result vary?
The answer, is the AsParallel() operation (which I explain in the Testing section of the report), which breaks up the list into parts to be processed on separate threads in parallel. The results are then brought together again. How quickly each thread gets through its assignment can vary according to what other background processes are running on the machine. Therefore the partial lists may complete their processing at different times - and are presented to the ‘aggregator’ in a different order.
This hadn’t, I confess, occurred to me. So I’ve learnt something very new from this project. It doesn’t affect the effectiveness of the solver, but I need to be aware that if I am going to use AsParallel(), then I need to manage the conditions in the Aggregate (Fold/Reduce) functions more carefully if I want the system to be fully deterministic.
Now I need to write several paragraphs of justification to students submitting 1-page of if-statements as code - just why their submissions aren’t worth 75/75 marks
I think it’s time to move away from marking these things - to simply applying a “pass/fail” science practical assessment to them.
Thanks - I will be in touch when you’re back, there’s no rush
I’ll add it the Site issues forum.
Enjoy your trip away!
Thank you, Simon. I’d be happy to run a webinar on one or more aspects of this, if you can help arrange. Will have to be March now as I am going to be overseas for all of Feb and the time difference would make a live webinar difficult. But I will still be active online and will be aiming to publish more blog posts relating to this area throughout Feb, or maybe before end of Jan.
On another matter, Simon, please could you use your contacts to get this message to the right person in CAS: I tried to edit my post to remove some typos, but it tells me (even when trying to delete some text) that I have ‘exceeded the 4000 character limit’. I think this constraint used to apply to the posting itself but was removed when I and maybe other ‘windy’ contributors protested. But they forgot to drop the constraint on Edit also.
Brilliant! Brilliant!
As ever, Richard, a really valuable contribution, thank you not just for the proof of concept that less may indeed be more but also for the insights into how FP is becoming de rigeur. As an aside: I recall being asked back to do some A Level teaching a few years ago with the newly added FP topic and was sinking well before swimming and I’m not convinced I managed even a few “strokes” (to persist with the unfortunate metaphor). I’m a fan of the Advent of Code puzzles and viewing other solutions (in C#) to those problems see the FP approach, using LINQ, as an often used approach.
The report you have painstakingly written is REALLY helpful - I hope other A Level teachers will download, view and use as exemplar.
I, for one, am looking forward to seeing more …
and … would you be willing to run a webinar on the process? It’d be helpful for other A Level teachers when prepping for next year’s students.