Journal

Chapter 3 – Assessment

Mercer Review: Chapter 3 – Assessment

1. Introduction

1.1. The Mercer Review Call for Evidence asked a number of questions regarding the assessment of a UK patent attorney:

• Q1b) Do you have any comments about the administration of the PEB examinations?
• Q2a) Do you have any comments about the current knowledge, understanding and practice covered by the Foundation Certificate examinations?
• Q2b) Are there any additional areas of knowledge, understanding or practice that should be covered by the Foundation Certificate examinations
• Q2c) Are there any areas of knowledge, understanding or practice that should be removed from the Foundation Certificate examinations?
• Q2d) Do you have any other comments about the Foundation Certificate examinations, for example in terms of language, timing, subject matter or training period?
• Q2e) Do you have any comments about the use of university qualifications, such as those provided by Queen Mary University of London, Bournemouth University and Brunel University, as an alternative to the Foundation Certificate?
• Q3a) Do you have any comments about the current knowledge, understanding and practice covered by the Final Diploma examinations?
• Q3b) Are there any additional areas of knowledge, understanding or practice that should be covered by the Final Diploma examinations?
• Q3c) Are there any areas of knowledge, understanding or practice that should be removed from the Final Diploma examinations?
• Q3d) Do you have any other comments about the Final Diploma examinations, for example in terms of language, timing, subject matter or training period?
• Q4a) Do you have any comments about the use of closed-book examinations for the Foundation Certificate and Final Diploma examinations?
• Q4b) Are there any other approaches to assessing knowledge, understanding and practice that should be considered in addition to, or as an alternative to, closed-book examinations?
• Q4c) Do you have any comments about the qualifications listed in Schedule 3 of the IPReg Rules for the Examination and Admission of Individuals to the Registers of Patent and Trade Mark Attorneys leading to part or full exemption from the Foundation Certificate examinations?
• Q6a) Do you have any comments about the requirement for newly qualified patent attorneys to obtain a qualification in Litigation Skills within three years of entry onto the Register?

2. Methodology

2.1. This sub-group was tasked with reviewing the responses to these questions. More generally, we considered whether the examinations that a UK patent attorney will sit are suitable for assessing the skills and knowledge a UK patent attorney ought to have.

3. Overview

3.1. In the previous chapter, we suggest several recommendations and reasons for changing the PEB syllabus. In this chapter, we further reviewed the assessment of patent attorneys.

3.2. For example, we recognised that the content of the examinations should be adjusted so that any ‘creep’ towards more complication and more content is reversed.

3.3. It is also important to look at the way in which the examinations are taken and assessed. In particular, the effect of the change to electronic examinations in light of the Covid-19 pandemic for both the PEB examinations and the EQE and the proposals for changing the format of the EQE led us to consider the format for the PEB examinations.

3.4. A further point to consider is the timing of the examinations. Candidates and their employers are placed under pressure by having to juggle the EQE qualification route with the PEB qualification route. It therefore needs to be considered whether there are ways to organise the examinations to reduce this pressure.

3.5. We also recognised that, in looking at the examinations, there were also implications for training and the way in which the examinations are set.

3.6. One aspect of training which led to considerations relating to the examinations was the fact that the ways to qualify to take the FD examinations does not consistently lead to candidates learning in all the areas covered by the FD examinations and the LSC. It was considered that a way to ensure that all candidates have learnt in the relevant areas would be to make it compulsory for all candidates to pass the PEB FC examinations (see above).

3.7. It was considered that the PEB FC examinations are generally appropriate but could be improved by ensuring that the syllabus covers all the reserved legal activities covered by the Legal Services Act and all the ‘black letter’ learning about the UK litigation system which is at present part of the LSC. The law on evidence (as opposed to the drafting of evidence) was highlighted as an important point. It was also considered that the FC examination syllabus should cover ethics.

4. Foundation Certificate

4.1. It was noted that candidates for the EQE are required to register with the EQE Secretariat as soon as they join the profession. It is considered that it could be advantageous to require any person intending to take the FC and FD examinations to register with the PEB as soon as they decide to take the examinations. This would allow the PEB to direct the candidates to sources of training. It could also be required that, for entry to the FC examinations, the PEB could require each candidate to provide evidence that they have either been under the supervision of a registered patent attorney for at least one year or followed a suitable training course and had two years of experience in an IP environment.

4.2. We recognise that students do not all have the same training opportunities and resources at their place of work; the variation in size and nature of the firm or department makes this inevitable (see above). We also recognise that employers value the availability of different training routes, to suit their different circumstances. It is clear, however, that the current system leads to the situation where trainees can qualify to take the FD examinations with differing levels of knowledge and experience. Furthermore, these differences may impact on how prepared the candidates are to sit the Final Diploma and therefore their performance in those examinations.

4.3. The proposal that the PEB FC examinations should be compulsory led to further considerations. It would need to be ensured that the PEB Foundation examinations are of high quality and cover an appropriate syllabus. It was envisaged that IPReg would need to be involved in setting the syllabus and ensuring the quality of the examinations. This could lead to a simplification of the system for setting the examinations and therefore for there to be more transparency.

4.4. There was a concern that making the PEB FC examinations compulsory could have an adverse effect of access to the profession. It was thought that this could be dealt with by making the FC examinations open to anyone who has taken a suitable course or has been working for a registered UKPA for at least one year. Thus, a candidate not employed by a UKPA would be able to take a course, such as the ones offered by QM-UL, Bournemouth University and Brunel University, and then take the FC examinations without needing to be in employment.

4.5. There may be an advantage to IPReg in this proposal as it would support IPReg in the accreditation of courses, whether academic or commercial. All IPReg would need to do is to inform any course provider of the syllabus for the Foundation examinations and to receive in return a commitment to teach to the syllabus. It would then be up to the provider to ensure that the teaching followed the syllabus. If the provider did not teach to the syllabus, this would be reflected in the results obtained by the candidates who took that course. Market forces would work either to eliminate inadequate courses or to incentivise inadequate courses to improve. It may also enable other course providers to enter the market.

4.6. This could also be an advantage for course providers as they would need only to provide teaching and not to provide examination. This should reduce the cost of the courses and so not add to the financial burden of candidates or their employers.

4.7. The question of training arose from these considerations. As noted in Chapter 2 above, it was recognised that, even for candidates who are employed, there is a great variety in training. Some firms have in-house training schemes which prepare the candidates for the examinations but others have almost no in-house training and rely on outside providers or expect the candidates to train themselves. It was felt that information on what sort of training is required, how it may be acquired and what to look out for when applying for a job should be available to students and others thinking of entering. We noted that the Informals is active in this area and we need to co-ordinate the work being done by the Informals and the Training Group within CIPA, hence the recommendations given in Chapter 2.

5. Final Diploma

5.1. As to the FD examinations, it was considered that these are generally on the right lines. However, it was felt that examination creep had taken place and so the examinations in recent years have become over-complicated and too long. It was considered that each examination should be limited to its core area. Thus, FD2 should be limited to drafting a patent application suitable for filing at the IPO in a form where the claims are clear, novel and arguably inventive over the prior art presented in the question and where the description is sufficient. FD3 should be limited to answering an official letter from the IPO which raises novelty and inventive step objections and providing a set of claims which deals with the objections and which does not add matter or lack clarity. FD4 should be limited to requiring the candidates to demonstrate that they can construe a set of claims, evaluate prior art, determine whether the claims as construed are novel and inventive over that prior art and determine whether the activities of a potential infringer are infringing acts, all according the case law in the UK. FD1 should not cover any of the areas covered by the other examinations but should require candidates to show that they have the knowledge of UK law and can apply that knowledge in a commercially relevant situation. FD1 should also include at least one question about a situation which could arise in litigation of a patent in the UK courts.

5.2. There have also recently been increases in the time available for the FD examinations. For example, FD4 increased from four hours to five hours in 2010, and this extension in time seems to have led to an increase in content. More recently, due to the Covid-19 pandemic and the nature of online exams and requirements for screen breaks, the time available for FD4, was extended to nearly six hours (without including any extra time for those with reasonable adjustments). We consider that the ‘inflation’ of papers should be reversed. It was noticeable that the FD4 paper for 2020 was shorter than previous papers and the pass rate was significantly higher. This is a trend which should be maintained.

5.3. There was a discussion of whether the availability of marking schedules for the FD examinations was appropriate. It was considered that this may detract from providing a holistic answer. For instance, in FD4, it was considered that giving a mark for telling the examiner what ‘comprising’ means or for explaining the meaning of a word which makes no difference to the answer means that some candidates spend too much time on mark accumulation and not enough time on working out a holistic answer to the whole question. It was therefore questioned whether marking schemes are useful for candidates. It was, on the other hand, pointed out that marking schemes are better than nothing and so, if they were not to be published, it would be necessary for the examiners to provide more specific feedback on what they were expecting, how candidates did well and how candidates did badly. As discussed above, this could be achieved by a training-the-trainers session on each examination as soon as the results come out.

6. Open vs Closed Book Exams

6.1. The question of open- vs closed-book examinations was discussed in the responses to the call for evidence. There was a fairly even split between those favouring closed book examinations, in particular at FC level, and those who felt open book examinations were more reflective of the real world. On the one hand, closed book examinations were seen as helpful for embedding information; on the other, it was felt that the FC syllabus is so broad that it is simply a memory test, resulting in candidates cramming the material rather than seeking to understand it. Closed book examinations were seen as less appropriate at the FD level.

6.2. The discussion is also influenced by the change, which may well become permanent, from paper to electronic examinations. Taking the examinations electronically is closer to real life, in that almost all candidates at work use a PC or laptop and therefore have many resources available on-line. It was considered that, as electronic examinations are closer to real life than paper examinations, it would be sensible to allow some access to sources electronically. However, unfettered access to sources would not enable a proper test of candidates to be carried out. It was suggested that the candidates should be able to access the Patents Act (perhaps in the form of the Black Book), the EPC, the PCT and the associated rules.

6.3. If the examinations were to revert to paper examinations, then the PEB should make available these aforementioned open book sources as clean paper copies. If the examinations remain as electronic examinations, the PEB should make the open book sources available in read-only format as part of the electronic examination system to prevent copy and paste of source material.

7. Electronic vs Paper Exams

7.1. It appears that the switch from paper to electronic examinations was met with approval, although there are areas where development is needed. At the same time, the EQE was also switched to electronic format and it appears that the EQE Supervisory Board, with the assistance of the EPO and epi, is intending to continue with the electronic from now on. In 2022 and 2023, the EQE will retain substantially the present format but, in 2024, it seems there will be a complete change in format. It is considered that the system used for the EQE in 2021 was reasonably effective, in particular in terms of invigilation. It therefore seems appropriate for the FC and FD examinations to remain in electronic format. As the system used for the EQE provided audio and visual invigilation and otherwise appears to be adaptable to the FC and FD examinations, it would seem to be appropriate to use that system for those examinations. Thus, candidates would only need to get used to a single system.

7.2. A problem with an electronic format for examinations is that of general and screen fatigue. It is not recommended that candidates spend up to six continuous hours working on a single screen – this typically does not occur in a working day and so the exams should not require this either. It is therefore considered that the maximum time for any of the examinations should be four working hours. There could be compulsory break times during the examinations and the system should automatically upload the candidates work at a fixed time with no requirement for the candidate to do the uploading. It is suggested that, for a four-hour examination, there would be a compulsory break of fifteen minutes, during which time the screen would go blank so that candidates cannot add to their answers during this time. Similar arrangements should be made for shorter examinations.

7.3. An alternative is to divide the examinations into discrete sections, as occurred with the 2020 EQEs. This would avoid candidates having to take an enforced break potentially in the middle of a question and then having to return to the same question to finish it. For example, FD1 sections A and B could be separated so that the candidates can take a more natural break between the two parts.

7.4. In order to keep the examinations to a maximum length of four hours, it would be necessary to ensure that the papers are short enough for a candidate to have a reasonable possibility of finishing the examination in the available time. The examinations should also be designed such that there is no requirement for providing in the answers comments on points which are irrelevant to the overall answer.

7.5. It was recognised that putting the examinations in electronic format has its drawbacks as well as its advantages. Care would have to be taken to ensure that no candidate is disadvantaged by the use of an electronic system. It therefore may be necessary to enable candidates who would otherwise be disadvantaged to take the examinations in paper form or to take the examinations at a special site.

7.6. The responses to the call for evidence generally did not see a need to change from the present examination system to a university-based examination system or a continuous assessment system, although there were a few responses in this direction. We considered that an examination system is appropriate for assessing whether a candidate meets the standard for registration and that the other assessment systems were not appropriate as they would not lead to a uniform assessment process or would be too expensive or time-consuming.
.

7.7. If the examinations were to revert to paper examinations, it is considered that the present format is suitable but the same points as regards the syllabi, content of the examinations, time for each examination and marking for the examinations made above also apply. In this case, however, the venues for the examinations should be improved, whilst maintaining good accessibility. For example, the Crypt has been described as small, noisy, and dark, which may not provide ideal examination conditions.

8. The Litigation Skills Certificate

8.1. It was considered that, as long as each candidate takes the FC examinations and they and FD1 cover the ‘black letter’ and tactical aspects of litigation, as presently covered by the LSC, there may be no need for the LSC to be a compulsory part of the qualification for entry onto the Register. However, it was considered that the ‘advocacy’ part of the LSC should be made an available optional post-registration training for those wishing to specialise in oral proceedings before the EPO or conducting litigation before IPEC or, if further qualified, before the High Court and above.

9. Other Examinations

9.1. It was questioned whether there was a need for examination beyond registration. It was generally thought that this was not necessary. This would not necessarily apply for the Higher Court Litigation Certificate or the Higher Courts Advocacy Certificate. The only other question was whether there should be an examination route to entry on the trade mark register for UKPAs. It was also questioned whether UKPAs with suitable experience should be allowed onto the trade mark register without examination.

9.2. It was considered that, if a registered UKPA wishes to specialise in a particular area or areas, such as becoming a design attorney or specialising in Supplementary Protection Certificates, for example, there should be no requirement for examination. However, there should be provision of training in specialised areas which should be acknowledged, for instance by way of a certificate.

10. The Timing of The Exams

10.1. There are issues with the timing of the examinations. Most candidates enter the profession in the autumn of year 0. They are unlikely to attempt the FC examinations in the same year and so most candidates will take the FC examinations in autumn/winter of year 1, after about one year in the profession. If our recommendation is accepted, this might require any training providers to adapt their training schedules to an autumn/winter examination.

10.2. If a candidate passes the FC examinations in year 1, they are entitled to attempt the FD examinations in November of year 2, after about two years in the profession. It is considered that, for many candidates, this may be too early to take all the FD examinations as they may not have received full training or gained enough experience for all the FD examinations in those two years. Also, in the February/March of year 3, they will usually attempt the pre-EQE examination.

10.3. If a candidate does not attempt the FD examinations in November of year 2, they may attempt them in November of Year 3. However, assuming that the candidate passed the pre-EQE, they will likely be attempting the full EQE in February/March of year 4.

10.4. Thus, each candidate has a few very full years of preparation for and taking examinations. This can lead to examination overload and can also lead to confusion between UK law and EPC law, especially as regards inventive step.

10.5. It seems unlikely that any changes to the schedule for the UK examinations and EQE can be made in the near future. It may be that, in the future, the whole format of the EQE will change, which may relieve the examination overload. For the present, we note that both the UK examinations and the EQE are modular. Effective use of the modular system can lead to less examination pressure.

10.6. It was questioned whether the examinations could be set twice a year. This was attempted previously but the pressure on resources and the limited number of candidates meant that the attempt did not succeed. It may be possible, if the UK examinations remain electronic and modular, for there to be two sittings a year. However, the pressure on resources could again be a problem. This may be limited to some extent if one of the sittings was limited to re-sitters. However, that may not give resitters time to prepare for the resit, even if the results were available within three months of the first sitting.

11. Exemptions

11.1. As noted above, there are proposals for changing in a radical way the format and timings for the EQE. In the responses to the call for evidence, questions were raised about the fact that a candidate does not need to take FD2 and FD3 if the candidate has passed the EQE. It was pointed out that it is possible to pass the EQE while not passing Papers A and B. A candidate could obtain 45 marks each for Papers A and B (thus failing papers A and B) but obtain 55 marks (or more) for each of Papers C and D, thus accumulating at least 200 marks and obtaining an overall pass. No such ‘compensable fail’ system is used in the FD examinations and so it was questioned whether it was appropriate for an exemption from FD2 and FD3 to be granted for candidates who only achieved a compensable fail for Paper A or Paper B. It was also indicated that the view was that Papers A and B of the EQE are less stringent than FD2 and FD3.

11.2. The situation will be complicated by changes to the EQE which may be implemented by 2024. It may be that the stringency of examination of the subjects of FD2 and FD3 in the EQE will be further reduced and so the exemptions may not be justified. In addition, there may no longer be a direct comparison of papers.

12. Conclusions and recommendations

12.1. In light of the above, we therefore recommend that:

• • IPReg, CIPA and the PEB investigate whether early registration of candidates should be implemented.

12.2. We also recommend that:

• qualification for the Foundation Certificate should be via the PEB FC examinations, with all course providers teaching to the same syllabus.

12.3. We also recommend that:

• the PEB has as a continuing task of ensuring that the content of each FD examination remains limited to its core area; and
• the PEB has a continuing task of ensuring that the length of all the FD examinations is maintained in a defined size range from year to year.

12.4. We also recommend that:

• the PEB does not make available marking schedules (as occurs with the EQEs) to candidates but provides more detailed examination reports and provides train-the-trainer sessions immediately after release of the results for any FD paper; and
• the PEB should make it clear that, for each of papers FD2, FD3 and FD4 and for each questions in paper FD1, the examiners are looking to see whether the answer as a whole merits a passing mark so that candidates do not concentrate on ‘mark gathering’.

12.5. We also recommend that:

• when taking the FC and FD examinations, candidates should have read-only access to a limited selection of sources to be determined by the PEB.

12.6. We also recommend that:

• the FC and FD examinations should continue to be held online and, if appropriate , should use the same system as is used for the EQE.

12.7. We also recommend that:

• the PEB, together with IPReg and CIPA, should investigate the use of the electronic examination system used for the EQE to see whether it can be adapted to meet the requirements of the FC and FD examinations and allow read-only access to selected sources;
• the PEB should adapt the examinations and marking schedules as necessary so that the maximum working time for any examination can be limited to four working hours, excluding any additional time that is required for e.g. students with reasonable adjustments, breaks, or uploading/downloading time; and
• the PEB and IPReg should consider whether the invigilation system used by the system is sufficient.

12.8. We also recommend that:

• the PEB, IPReg and CIPA should encourage employers to support their candidates in effective use of the modular examination systems without affecting career progression; and
• the PEB, IPReg and CIPA should look at the scheduling of the UK examinations once more is known about how the proposed changes to the EQEs will be implemented in 2024 and beyond, so as to avoid potential clashes.

12.9. We also recommend that:

• the PEB, IPReg and CIPA should investigate whether having two sittings a year is a practical proposition.

12.10. If, as expected, the UK examination system remains in electronic format, we recommend that:

• the examinations should be spread over two weeks so that there is a gap of a day between each FD paper and there is only one FC paper per day.

12.11. We also recommend that:

• IPReg, with the assistance of the PEB and CIPA, should investigate whether the exemptions from FD2 and FD3 in light of a full pass of the EQE are appropriate.

12.12. As noted at the end of chapter 3, any changes should be reviewed to ensure that, as far as possible, they encourage an increase of diversity and inclusion in the profession.