School Syllabi and Assessments

Before discussing the analysis, it is helpful to contextualise the major educational systems our students come from:

1. 1) A Level Latin is ordinarily sat by pupils in the final year of secondary school (aged 18). It is usually one of a three- or four-subject one-year course of secondary school study (changing to a two-year linear course from 2018). It requires 360 hours of ‘guided learning’. Typically a set text prescription of 300 lines of verse plus prose equivalent is set; language assessment is worth 50% of the final grade.

2. 2) IB (Higher Level) Latin is part of a broad scheme of study for students aged 16-19, which narrows in the final year to the study of three or four subjects. It requires 240 hours of ‘guided learning’.Footnote ¹ A set-text prescription of c.450 lines of verse plus prose equivalent was set in 2013 and of c. 500-550 lines from 2016. Language assessment is worth 35% of the final grade.

3. 3) ‘Highers’ Latin is a Scottish level 6 qualification taken by pupils aged 16-17, normally as part of a five-subject course of study, and is traditionally the last qualification required before entrance to university. It requires 240 hours of ‘guided learning’. Language assessment is worth 40% of the final grade. Instead of continuous prose a more varied selection of prose and verse is offered for the ‘set text’, amounting to approx. 220 lines of verse plus prose equivalent.

4. 4) ‘Advanced Higher’ Latin is an optional qualification (Scottish level 7), taken by pupils who have completed Highers and chosen to stay on at school for an extra year. It is normally part of a three-subject course of study. It requires 320 hours of ‘guided learning’. Set texts offer a range of authors in verse and prose (approx. 440 lines of verse or prose equivalent). Language assessment is worth 30% of the final grade.

Choosing randomly, we selected the years 2013 and 2016 for analysis, our sample comprising eight A Level exams, four IB exams, two Advanced Higher exams and five Higher exams (2013), with one exam from each type of qualification (2016) for comparative analysis.Footnote ² Scrutinising not just the syllabi but also past papers and marking schemes, we paid special attention to the following areas:

1. a) Type of exams and exam questions

2. b) The knowledge of grammar (morphology and syntax) required

3. c) The knowledge of vocabulary required

4. d) The marking scheme

Comparison of these categories reveals telling divergences in the ‘philosophy’ of language teaching and testing and levels of linguistic competence expected across A Levels, IB, Higher and Advanced Higher.

School Teacher Questionnaires

Syllabus and assessment patterns do not tell the whole story, of course. To gain a clearer understanding of how these courses are delivered, we sent questionnaires to different types of schools (state and independent), to teachers delivering a range of qualifications. We asked about pedagogic practices (e.g. how much class and homework time they prioritised for grammar teaching/revision, unseen translation, vocabulary learning; which textbooks they used, etc.), and about the expectations teachers had of University-level Latin language teaching.

Predictably, there was huge variation (both at a personal and at an institutional level) in approaches to Latin language teaching: some teachers teach unashamedly to syllabus (‘there's no gerundive in the assessment, so I won't teach it in class’), and some passionately eschew that approach in favour of a more rounded model: ‘I teach them what I think they need to know, and that goes beyond what they will be tested on.’ There was also huge variation in the balance and spread of activities prioritised for class-work and homework, even within the same qualifications. Some teachers clearly spend the majority of their time on set text translations and/or literary discussion; some on unseen translations (while some do very little of that); some on grammar exercises; some on comprehension exercises; some regularly test vocabulary, some never do. Variation in teaching approaches is inevitable across all disciplines and not a problem in itself; but it came across very prominently in these questionnaire responses, and serves as a useful reminder that differences between students’ Latin language-learning experiences are not simply down to differences between the syllabi that they have been following.

Less predictable (to us, as university lecturers) was the fact that a significant number of respondents were quite unclear about what university-level Latin courses expect of first year students. In fact, a very small minority suggested that that since very few if any of their pupils went on to study Classics/Latin at university, there was little call for them to know what happens in first-year university-level Latin classes. Conversely, many university colleagues have only limited awareness of the preparation which most entry-level students currently receive. This underlines the need for even greater communication between school and university Latin teachers, in both directions.

Student Questionnaires

In their first week at St Andrews, three successive cohorts of students were given a detailed questionnaire covering their educational background, their sense of their own linguistic strengths and weaknesses, and their expectations of Latin study at university.

Even within the same qualification type, participating students had clearly had hugely varied experiences of Latin language-learning, resulting in different levels of confidence in different areas. Encouragingly, though, the majority of students felt that school had prepared them adequately both for their school-level qualifications and for studying Latin at university. No recurring issues arose consistently within any one cohort about specific areas of Latin language work, with the exception of a prominent and repeated concern articulated by Advanced Higher and Higher students that they did not have enough memorised vocabulary ‘in the bank’. The areas in which they thought they would need particular support during their first year at university were evenly spread between all possibilities, ranging from support with literary analysis to revision of grammar. One general anxiety was articulated by respondents from all cohorts, namely the challenge of bringing an abstract command of grammar and vocabulary together with the study of original texts. This expectation was encouraging, in so far as it suggested that students across the board clearly recognised that applying linguistic knowledge to reading and analysing texts independently would require special attention and effort on their part, given the limited experience they had in doing these activities at school.

We also ran much shorter questionnaires for students to fill in at the end of their first year, combining analysis of these with the data from institutional module feedback forms. Two recurring trends are particularly noteworthy. While the incoming questionnaire responses revealed some insecurity amongst a wide spread of students about their own language competence and their ability to keep up, the end-of-year questionnaire responses painted a more confident and settled picture. Perhaps unsurprisingly, there was a sense that by the end of the year students had adapted to the expectations of the course and felt less hesitant about being able to meet them. The concerns that remained tended to revolve around ‘structural’ issues (library provision/deadlines etc.), and not about the intellectual challenges of university-level work or their ability to manage it. On the other hand, we noted a slight shift from industrious intentions to a stronger sense that problems/difficulties stemmed more from institutional issues than from individual circumstances. There was notably less emphasis on the need for on-going independent effort: a misapprehension, and a reminder that our students need to be challenged as well as supported if they are to continue making progress.

Diagnostic Test Data (2013–4)

Students also sat a diagnostic test in their first week of study, comprising four parts: a) vocabulary, b) verb parsing, c) declension of adjectives and pronouns, and d) a gap text asking students to enter the correct morpho-syntactical form of the word indicated in brackets. The aim of this test was to compare students’ competence in each of these areas with the answers they provided in the questionnaires, as well as to provide a base-line from which performance in first, second, third and fourth year university Latin modules could be analysed.

Diagram 1 shows students’ overall performance in the diagnostic test (scaled to fit the St Andrews 20-mark system) in accordance with school qualification, i.e. A Level (AL), IB, Higher (H) and Advanced Higher (AH):Footnote ⁴

With one exception in A Levels, this test result showed Higher and Advanced Higher pupils performing significantly less well than A Level and IB pupils. Yet a close look at the results by category reveals a more complex picture. Diagram 2 shows the results obtained in each component, with blue showing vocabulary (Voc), orange verbs (Verbs), grey adjectives and pronouns (A/P) and yellow the gap test (Syntax), and with qualification types indicated below:

• - With one or two exceptions, Highers and Advanced Highers were the weakest in the vocabulary section. This result correlates with the anxiety expressed in student questionnaires and the lack of emphasis on vocabulary learning in the syllabus.

• - The A Level and IB cohorts also outstripped Highers and Advanced Highers students in the ‘Verbs’ category.

• - In the ‘Adjective and Pronoun’ section Higher and Advanced Higher students produced low scores again, but there was also much greater variation in performance at IB and A Level.

• - In the final section, where active knowledge and understanding of syntax was tested, the results were most varied and indicate a shift in the pattern so far established: despite the fact that the highest results were achieved by A Level students, the overall scores obtained in the syntax section are much more level between the four cohorts.

It may be a statement of the obvious that Highers and Advanced Highers will perform less well than their A Levels and IB peers at the point of entering university, since on average they have received fewer teaching hours and their syllabus places no real stress on the memorisation of a basic vocubulary. But it is also important to note that a significant minority of A Level and IB students lack fairly basic language competence, and further that a significant majority who scored highly in the preliminary (memory-based) sections of the diagnostic test did not cope so well in sections testing more active engagement with and understanding of Latin.

First Year Students’ Data (2013–4)

The data we have looked at so far records the linguistic experience and competence of our Latin students up to their first week at university. But we wanted to see whether the picture changed during the first year of Latin study, and over subsequent years, and to find out whether the data could offer a more informed picture about the kinds of support we need to offer our students in the first week, the first year, and over a four-year period.

In their first year of ‘Advanced’ (as opposed to ‘Beginners’) Latin study, St Andrews students are required to complete two modules (coded LT1003 and LT1004). Both consist of the following assessments: i) two ‘literary criticisms’, of one prose and one verse set-text; ii) two language tests, one ‘unseen translation’ and one ‘grammar, morphology and syntax’. These components are equally weighted and account for 40% of the marks. In addition, an end-of-semester exam, comprising 60%, tests both translation and literary criticism of the set texts.

Literary Criticism

Diagram 3 shows the overall results in the first semester module LT1003 (LT2003Y2) in comparison with the diagnostic test results (DT2), with qualification type indicated below:

Strikingly, there is now no gap between the overall LT1003 performances of Higher and Advanced Higher students, while A Level and IB still show wide variation.

Analysis of the individual components of LT1003 provides further detail. Diagram 4 shows students’ performance in the module's two literary criticism components (on Catullus, LC1 Cat, and Cicero, LC2 Cic), mapped against their diagnostic test results (DT2):

Even though some variation in performance between the two literary criticism tasks is visible, with few exceptions, students of all cohorts achieved higher results in the literary criticism assessments than they did in the diagnostic test, where linguistic competence was the focus. The tendency for students to perform comparatively well in literary criticism is confirmed in the second semester module LT1004. Diagram 5 shows the overall results obtained in the second semester literary criticism component (LT2004Y2), mapped against diagnostic test results (DT2):

As for LT1003, the literary criticism marks here are more consistent across different cohorts within our small sample than the diagnostic test marks.

Language and Unseens

In the language component of our first year modules, students are assessed on both their theoretical and practical knowledge of Latin. The former is evaluated with the help of a language test, the latter with an unseen translation test. Diagram 6 shows the results obtained in the language (LT1003Y2 LT Lang) and unseen tests (LT1003Y2 Unseen) for the first semester module LT1003 (with diagnostic test results for comparison, DT2):

A significant minority of students obtained worse results, relatively speaking, in the language components of the first semester module than in the diagnostic test: a fact especially true of A Level students, whose school qualification put most emphasis on ‘unseen’ translation. Highers and Advanced Highers on the other hand showed significant improvement in their language performance when compared to the diagnostic test result (though their results are still in the bottom half of the language test). Strikingly, and as we saw too in the diagnostic test, when it came to the more ‘active’ component of Latin assessment (unseen translation), Highers and Advanced Highers did as well as, and in some cases better than, their A Level and IB peers. These results demonstrate clearly that knowledge in the abstract and ability to apply that knowledge to ‘real’ Latin do not necessarily correlate.

Introducing the Linguistic Structures element

In response to the findings of the first year of the SELF project, in which a clear gap between students’ engagement with abstract grammar and active Latin was already noticeable, we decided to introduce a ‘linguistic structures’ class (LS), in which our aim was deliberately to break the normal student experience of ‘set-text’ preparation and to connect ‘theoretical’ knowledge more obviously with close reading. Focusing on a selection from Cicero's Pro Archia, students were explicitly not required to translate in normal ‘set-text’ fashion. Instead, each hour of teaching was devoted to a specific linguistic topic, with a cumulative structure allowing students to see connections between linguistic phenomena. As pre-preparation, students were tasked with revision of the linguistic topic in question, before searching for examples of that topic within a specific passage of Pro Archia. Once in class, brief revision of the topic in question was followed by an in-depth analysis and discussion within the passage prepared independently. Over the course of the semester, students were also asked to ‘read backwards’ or ‘re-read’, identifying more complex clauses in chapters they had previously covered. We hoped that this approach would have the following benefits:

1. 1. A ‘level playing field’, with all students reading a new text and learning a method of analysis probably fresh to all.

2. 2. The opportunity for all students to learn a coherent and consistent method of ‘active application’ of abstract grammar to real Latin.

3. 3. A genuinely ‘in-depth’ reading of the set-text, through repeated re-engagement, introducing students to methods of independent revision and inculcating the importance of an ‘in-depth’ approach to reading primary texts.

4. 4. A way to enthuse students to engage more deeply with the structure of the text as a fundamental (rather than separate) feature of exploring ‘literary’ issues.

While we cannot gauge precisely how effective the new linguistic structures course is – it is, after all, just one element of the Latin-learning experience we offer at St Andrews – we do see signs that the clear gap between ‘literary’ and ‘linguistic’ competence is narrowing, as the results of LT1004 show. Diagram 7 compares diagnostic test results (DT2) with linguistic structures results (LT1004Y2 LS) and unseen test results (LT1004Y2 Unseen):

The stand-out cohort here is that of Higher and Advanced Higher students, whose performance in the linguistic structures test at the end of year 1 is strikingly higher than performance on entry (A Level and IB students tend to perform at a similar level to their diagnostic test entry scores). Partly this can be explained by preparation: this cohort has been prepared for the linguistic structures test in a way that they were not for the diagnostic test, and these students have had one year to work on vocabulary and grammar skills. Even so, there seems to be a clear knock-on effect for all students, with (barring a few exceptions) a closer correlation between performance in ‘language’ and unseen translation: the clear gap between abstract/theoretical grammar and active knowledge of Latin noticeable at the beginning of the year and at the end of the first semester is increasingly narrow.

This is further visible when comparing the overall language results of the first- and second-semester modules, as in diagram 8 (first semester = LT1003Y2 Lang av, blue; second semester=LT1004 Lang av, orange):

As can be seen, there is overall less variation in students’ performance in the language component at the end of the second semester than there was at the end of the first semester. Higher and Advanced Higher groups in particular are now scoring in the mid-range or top range of overall language results, but for the most part students from all cohorts are improving on their language scores in the second semester (see also diagram 9, where second-semester linguistic structures results – LT1004Y2 LS – are compared with first-semester language results – LT1003Y2 LT Lang):

The picture at the end of Year 1

The anecdotal perception that Scottish students need particular help in the first year of study has been justified by the analysis of school syllabi, which shows that these students enter university with many fewer hours of language training and little or no vocabulary knowledge. Yet while Advanced Highers and Highers students enter university with language skills that are demonstrably weaker than their peers, by the end of the first year of study they are firmly within the centre of overall performance (diagram 12). Furthermore, our study revealed significant weaknesses in other, more ostensibly well-prepared cohorts. A significant minority of A Level students’ performances dropped relative to performance in the diagnostic test over the course of the first year. Some were dragged down by performance in literary components of the course, while others failed to cope with the transition from tests centred on ‘abstract’ Latin knowledge to deconstruction of real Latin in the linguistic structures element (diagram 10 compares overall second-semester marks – LT1004Y2 – with diagnostic test results – DT2):

This is clearly a small sample, but results from other years (2012-2015) reflect the same trend. There must be at least the suspicion, then, that A-level students find it particularly hard to acclimatise to a largely independent learning experience, and that university teachers need to be more aware of the particular need this cohort might have for help and guidance in independent learning. It remains the case that students with Higher or Advanced Higher Latin need particular kinds of linguistic support, especially towards the start of their university studies; but the results at the end of the first year complicate the simple picture provided by the analysis of syllabus type alone, with the capacity to adapt to the independent learning experience emerging as one of the biggest factors for students’ successful acquisition and consolidation of new linguistic skills.

Second, Third and Fourth Year Data

Students were also asked to complete questionnaires at the beginning of their second, third and fourth years which concentrated on how they felt language competence and confidence had changed over the course of their degree. We found that while students of all cohorts often worried that they were getting worse at Latin during their first year, by the start of the second year the vast majority felt that their linguistic competence had advanced, at least a little. Second years were also more accurate in defining areas of weakness (for example identifying complex morphology and syntax as priorities for revision; asking for more unseen practice); and though they often asked for more support in the form of contact hours with tutors, there was greater recognition that they could work on weaknesses independently. Third and fourth year respondents were even more positive, with the vast majority rating their linguistic competence now as ‘good’, and asserting reasonable confidence in their ability to read extended and complex Latin in the original on their own and to interpret texts with some degree of sophistication.

While Highers/Advanced Highers students felt particularly anxious at the beginning of year 1, their worries had largely dissipated by the start of the second year, and their questionnaire responses offer the same kind of responses as other cohorts. The data bear this out. While the students we have focused on (entering 2013-4) have not yet graduated, historically those Higher and Advanced Higher students (particularly the latter cohort) who have continued with Latin through to the end of their degree have tended to graduate with the same range of final degree results as students from other educational backgrounds. (The four-year degree that is typical of Scottish universities may be a helpful factor here.) That said, our retention rates for Higher and Advanced Higher students across the degree course tend (worryingly) to be slightly lower than for other students (but comparable with our retention rates for students starting Latin from scratch at St Andrews); Higher and Advanced Higher students are marginally more likely to switch out of Latin modules over the course of their degree than their counterparts with other school-level Latin qualifications.

Conclusions and directions for travel?

Educational philosophy and government policy rarely stand still, as the references to changes made in Scotland (with the new ‘Curriculum for Excellence’), England (the much vaunted return of ‘rigour’ at A Level from 2018), and even the changes currently being made to the IB within this article have shown. Moreover, the changes in rules on admission policies in the higher education sector, with a trend to welcome greater numbers of overseas students from an ever-widening range of countries and the removal of caps on student numbers, will no doubt continue to broaden the range of experience and educational backgrounds of the students we welcome to embark on degrees in Latin (including those who choose to learn Latin from scratch at university). The aim of this study has emphatically not been to ‘judge’ various Latin school-level qualifications, make ivory-tower proclamations about the state of Latin today, or advocate for any particular philosophy of language learning. Rather, our aim has been to gain a better understanding of where our students are coming from, and what help they need in making the successful transition from school to degree-level learning. Perhaps the most reassuring general conclusion to be drawn – and one that should be emphasised to all students, from whatever background, on entry to university – is that regardless of ‘where you come from’, if you are engaged, attend classes regularly, consolidate the language tuition you receive at university in your own time and work hard at translating your set-texts independently (with support from teaching staff), our study proves that you will succeed and indeed that there is no barrier to graduating with the highest class of degree.

That said, it is clear that some cohorts of students arrive better prepared than others. As a Scottish university, it is our particular duty to make sure that the traditional Scottish path to university – via a Highers qualification achieved at age 17 – remains open to students who wish to study at St Andrews. This report has shown that at present (owing to successive changes within the Highers curriculum and assessment policies) these students are relatively ill-prepared in terms of linguistic competence for courses which demand from the very beginning the independent translation of large amounts of continuous Latin prose or verse, and which assess language skills via (among other things) unseen assessments. This is a problem for those students wherever they go on to study, and therefore a challenge for Classics departments across the UK and beyond. One solution (tried out by us and by colleagues in other university departments) is to advise this cohort of students to follow the ‘Beginners’ rather than ‘Advanced’ route on entry; but this is a very demoralising move for students who have been studying Latin at school for a number of years already and something that we at St Andrews work hard to avoid where possible. While our data show that Highers students can ‘catch up’ relatively quickly (and indeed have been prepared well for literary analysis and appreciation, elements that become more and more important over the course of most Latin degrees in the UK system), it is clearly incumbent on the SQA to listen more carefully to the concerns being voiced by school teachers and university lecturers, so that Higher and Advanced Higher students do not continue to be let down in the future.

Perhaps the more surprising conclusion that this report has drawn is that while our largest group of incoming students – ex A Level – can show very good levels of linguistic competence in the abstract, this competence does not extend to confidence in translating large amounts of original Latin independently, or in sophisticated literary analysis (arguably a consequence of narrow mark-schemes in the literary analysis papers, the fact that A Level students are not currently required to tackle substantial essay questions, and the way in which many are now conditioned to ‘work to the test’). It remains to be seen what effect the new OCR A Level, to be taught from 2016 and to be first examined in 2018, will have, but at first glance the revisions to this syllabus seem largely to have been directed at increasing the ‘rigour’ of the language elements of the course, while if anything set-text prescription quantities seem to have shrunk. And while students most often worry about their knowledge of grammar in the abstract, what they find most difficult in practice is the (largely independent) set-text translation and analysis. School pupils have been memorising their set-text prescriptions for decades, but given that our study has shown that many receive very dedicated translation help from teachers at school (e.g., literal translations dictated by the teacher, or translation guided by individually-numbered-word) it is no wonder that the experience of translating original Latin largely alone is a shock. This is a trend for school teachers particularly to reflect upon, in the context of course of the increasing pressures which they are under (in some qualification systems even more than others) to ‘get the right results’ for their students.

Our findings underline how important it is that colleagues teaching Latin at university across the whole of the UK (and indeed all around the world, not just in Scotland) are aware of how much the educational background and experience of all our students has changed, in terms of learning environments as well as syllabus and assessment patterns. Without compromising on our end-goals, we need to reflect regularly on the expectations that we have of incoming students and on the expectations that we communicate to them. On a practical level, that might involve (for instance) offering more concrete guidance on the business of translating alone as well as the setting of realistic set-text prescriptions (offering more guided use of, e.g., hyperlinked texts can turn reliance on published translations and internet resources into a learning opportunity rather than a problem). At St Andrews we have also attempted to put in place some specific measures in both our teaching and assessment practices to help with this difficult transition. In addition to the linguistic structures course described above, we offer extra dedicated language classes to those students who score most poorly on the diagnostic test which all of our first years now sit, and early indications are that this is having a beneficial effect. We continue to experiment with the Latin learning experience; our resident Language Teaching Officer, Dr Juan Coderch, has also instituted the use of spoken Latin, a technique which enhances familiarity with many aspects of the language and which (in its relative unfamiliarity for the majority of students) also serves as a useful classroom leveller.

While the communications that university teachers develop with their students are of the utmost importance, there are other important dialogues to be had, as we have tried to stress. Most of the outreach activities that take place between universities and schools focus on literary or historical topics; the relatively small numbers of language students (compared with those studying Ancient History and Classical Civilisation) leave Latin language work sidelined in this context. More sharing of best practice between school and university teachers in relation to Latin language learning (via student-based events and teacher-focused workshops and inset days) would filter productively through to students both in their final years at school and at the start of their university studies; it would also valuably increase visible support for this threatened subject, and for the teachers who campaign for its continued existence in school-level curricula. We have been heartened by some of the conversations we have had with qualifications authorities over the course of this project, but deeply concerned by others. It behoves school and university Latin teachers to join forces with increasing frequency to communicate our shared hopes and concerns to those involved at all levels in shaping the future of Latin syllabi and assessment in schools.