This is TikiWiki CMS/Groupware v1.9.11 -Sirius- © 2002–2008 by the Tiki community Sun 23 of Jul, 2017 [00:37 UTC]
Menu
Last actions

Errora EPSO 01

Errors EPSO 01
Recruitment
EPSO
print
Dear Michael,

I enclose and attach my full complaint to the Ombudsman on the EPSO CBT case.

Thanks for your support,
Jaime



From: e-form@ombudsman.europa.eu mailto:e-form@ombudsman.europa.eu
Sent: Tuesday, March 29, 2011 5:36 PM
To: ROYO OLID Jaime (EEAS-ASMARA)
Subject: EOWEB#7149 New complaint from: jaime.royo-olid@eeas.europa.eu



Your complaint has been submitted to the European Ombudsman. We will send you an acknowledgement of receipt within a few days.

NB - Please note that this e-mail was sent from a notification only e-mail address. If you wish to contact technical support, please use the link below:

Contact technical support

Sender
From: jaime.royo-olid@eeas.europa.eu
Date: Tuesday, March 29, 2011 4:36:15 PM CEST
EOWEB_COMPLAINT_ID: 7149

Complaint about maladministration
Part 1 - Contact information
First name: Jaime
Surname: Royo-Olid?
On behalf of (if applicable):
Address line 1: EU-Delegation to Eritrea
Address line 2: Marsa Teklai Street, Nr20/22
Town/City: Asmara
County/State/Province: Eritrea
Postcode: 5710
Country: Eritrea
Tel.: +291 1 12 65 66
Fax:
E-mail address: jaime.royo-olid@eeas.europa.eu

Part 2 - Against which European Union (EU) institution or body do you wish to complain?
European Personnel Selection Office

Part 3 - What is the decision or matter about which you complain? When did you become aware of it? Add annexes if necessary.
URGENT IN VIEW OF EPSO's 2011 GENERAL COMPETITION STARTING IN MID APRIL




Dear European Ombudsman,

Considering that:

a. In spite of the Ombudsman's decision on own-initiative inquiry OI/4/2007/(ID)MHZ concerning the European Personnel Selection Office (EPSO), whereby EPSO's refusal to grant access to candidates to the CBT questions taken as "an instance of maladministration", EPSO has continued to apply the same policy ever since and namely for the competitions run in 2010. I attach my various letters and appeals to EPSO requesting access to my CBT, namely justified on the basis of candidates claims that EPSO had 'neutralised' a number of CBT questions and thus making marking statistically unsound.

b. As a result of EPSO's lack of transparency, the CBT questions database (unrealistically claimed as floorless by EPSO) remains unchallenged despite candidates' recurrent complaints and appeals. However, as a requirement for the registration to the 2011 General Competition, EPSO has posted in its website from mid-March 2011 a self-assessment CBT consisting of 20 questions (10 verbal, 5 numerical and 5 abstract reasoning). As demonstrated in my letter to Vice-president Maros Sefcovic (attached and sent in Cc to the Ombudsman on 23/03/2011), among the 20 questions, 3 are objectively erratic and an additional one is unfit to assess candidates. This sample CBT questions extracted from the EPSO's database is undeniable proof that they are subject to a margin of error greater than the % number of candidates that are to be pre-selected. The questions can be accessed from http://europa.eu/epso/apply/today/action_c_en.htm?sa_comp_id=5267&lang=en

c. This problem could be sorted without financial incidence for EPSO by simply recognising the margin of error and allocating accordingly to the CBT a proportional discriminatory attribute. In other words: for EPSO's pre-selection to be statistically sound it cannot exclude candidates on the basis of % mark differences lower than the likely margin of error in the CBT database. Hence, a realistic approach would be that EPSO applies a simple pass or fail threshold mark of an order of magnitude of 80% thus allowing for 20% margin of error.

d. Every year over 60,000 candidates undertake EPSO CBT tests and are exposed to their erratic nature and to the aggravating factor that they cannot appeal. This is an unacceptable despotic attitude contrary to fundamental EU principles and profoundly damaging to the image of EU Institutions. A part from the frustrations caused on candidates excluded on fundamentally unsound methods, recruits confront thereafter an adverse reputation as well as legitimacy concerns over their recruitment.

e. EPSO is still on time to publish a Corrigendum for the 2011 General Competition

I would like hereby to put forward the following complaints to the Ombudsman:

1. The CBT questions database (now translated into 23 languages!!!) for the 2011 Open Competitions, is to be considered unfit to discriminate a % nr of candidates which is lower than a realistic and transparent margin of error therein. EPSO should:

o Urgently publish a corrigendum setting statistically sound marking which takes account of the margin of error ex-ante (i.e.: pre-selection CBT mark of 80%);

o If the above results in too many candidates being pre-selected then a jury should establish additional pre-selection requirements;

o Allow CBT questions to public scrutiny;

2. The fact that I was not pre-selected for the Open Competition EPSO/AD/177/10-EPA, Administrators, published at Official journal: C64 A 16/03/2010, for which I held candidate Nr:1589927, is to be considered a statistically-unsound decision since:

o I obtained 35/40 (87.5%) and the model 20-question CBT database published by EPSO contain a margin of error higher than 15%. In 2010 I could only believe but as from March 2011 I have proof to assume that the five questions a failed were likely erroneously set out/formulated/translated by EPSO;

o There were accounts from participants who claimed that in their mark sheet a number of questions had been 'neutralised' thus setting out different marking denominators to different candidates (Some were examined and marked on the basis of 40 CBT questions, and others on less). The evidence refered to by candidates has not been discredited by EPSO, who however claims that no questions were neutralised and that CBT. The margin of error has been in any case ignored.

EPSO should:

o Admit to the Assessment Centre all candidates who obtained 32/40 and above, thus reasonably neutralising a likely margin of error of 20%;

o Allow candidates to review the CBT questions they sat.

3. The fact that I was not allowed to obtain access to the CBT questions which I was examined on for the same competition EPSO/AD/177/10-EPA;

EPSO should:

o Share the CBT questions with candidates after the examination.

4. The fact that for Open Competition EPSO/AD/185/10 "the candidates who have been invited to attend an assessment centre session obtained a minimum total of 48.67 points in the admission tests". This is further evidence that the CBT (60 questions in this case) was object of some sort of unrevealed ex-post adaptations or 'neutralisations' since questions are marked right (1 point) or wrong (0 points) and overall marks are to be integers from 0 to 60. Since each candidate has a different set of questions, this CBT set unequal tests to candidates. EPSO has yet not replied to my questions with Ref: 20110307-C9VSF3ME posted to them on March 7th 2011.



Thank you in advance for your consideration,



Sincerely Yours,



Jaime Royo Olid

Part 4 - What do you consider that the EU institution or body has done wrong?


Has been using unfir pre-selection CBT questions in spite of the Ombudsman's decision on own-initiative inquiry OI/4/2007/(ID)MHZ concerning the European Personnel Selection Office (EPSO), whereby EPSO's refusal to grant access to candidates to the CBT questions they have taken "is an instance of maladministration", EPSO has continued to apply the same policy ever since and namely for the competitions run in 2010;
Has been using unfir pre-selection CBT questions despite the succesive accumulation of appeals from candidates thus worsening and amplifying the scope of the original problem as more and more candidate have been victims of an unfit system;
EPSO's non-acknoledgement of error in examining methods are a waste of resources both for thousands of candidates every year and for EU-Insitutions;
By acting erratically and unilterally denying the worth of independent and public checks and balances, EPSO has been damaging the image of EU Insitutions and particularly that of the European Civil Service.
Part 5 - What, in your view, should the institution or body do to put things right?
In general EPSO should:

Be required to comply with fundamental transparency principles such that the validity of its methods be independently and publicly scrutinised. Currently, EPSO has embarked on propaganda over its success yet unduly verified validity of testing (i.e.: it claims improvements ignoring the dozens of appeals of candidates to verify the validity of the CBT questions they have been exposed to);
Be required to set out statistically sound pre-selection marking and allocate to CBT realistic attributes such as akcnowledging that there is to be reasonably expected 15 to 20% margin or error.
Recognise that it is practically impossible to set out errorless CBT questions more so when these are thousands and translated to 23 languages;
Should refrain from experimenting on pre-selection methods (i.e.: Situational Judgement) and follow consolidated examples of international best practices. Thus, in consonance to any elite institution in the world, EPSO's pre-selection should allocate at least some weighting to candidates certified performance (language skills, academic achievements, relevant work experinece, etc...) rather than limiting pre-selection exclusively in "predict performance".


For the 2011 General Competitions EPSO should:

Urgently publish a corrigendum setting statistically sound marking which takes account of the margin of error ex-ante (i.e.: pre-selection CBT mark of 80%);
If the above results in too many candidates being pre-selected then a jury should establish additional pre-selection requirements in line with the reality of AD profile. Note that for 2010 General Competition AD5 recruits average age has been 32. Yet for 2011 they encourage second year students to apply. No work experience required and three year university degree is disrespectful to other staff. This is incoherent with Human Resources policies such as vis-a-vis Contrac Agents Category IV (who are often better qualified than ADs), JEDs (who require higher standards to be recruited), highly qualified ASTs and the recruitement for the "Stage" (which usually takes candidates with at least a Masters);
Allow CBT questions to public scrutiny.
Concerning the Open Competition EPSO/AD/177/10-EPA, EPSO should:

Admit retroactively to the Assessment Centre all candidates who obtained 32/40 and above, thus reasonably neutralising a likely margin of error of 20%;
Allow candidates to review the CBT questions they sat.
Concerning Open Competition EPSO/AD/185/10, EPSO should:

Publicly clarify how it was possible to set "...a minimum total of 48.67 points in the admission tests" as a qualification mark when CBT is meant to be marked with integers from 0 to 60.
Part 6 - Have you already contacted the EU institution or body concerned in order to obtain redress?
Yes (please specify)



I have attached my 2010 letters to EPSO addressed to EPSO's Head of Unit Mr Theo Duivenvoorde and Mr Gilles Guillard in Part 3 of the complaint form.
Concerning the 2011 e-mail and letter to Vice-President? Maros Sefcovic I have forwarded them to EPSOs Director on an e-mail dated 25/03/2011. I enclose the e-mail addressed to him below:
I also enclose my registerred e-mail to EPSO concerning the Translators competition EPSO/AD/185/1.






From: ROYO OLID Jaime (EEAS-ASMARA)
Sent: Friday, March 25, 2011 10:52 AM
To: BEARFIELD Nicholas David (EPSO)
Cc: ASHBROOK Michael (ESTAT)
Subject: Request for urgent measures to make EPSO pre-selection statistically sound
Importance: High

Dear Mr Bearfield,
Kindly find attached a letter I have addressed to Vice-president Maros Sevcovic on EPSO's CBT and pre-selection process. I understand EPSO is exposed to all kinds of pressures and demands and that the negotiations of all of them is a great challenge. So I sincerely hope my inputs are taken to the extent possible as critically-constructive feedback and that you find them of use particularly to avoid the same problem this year.
In addition, I would like to comment a couple of issues after having carefully viewed your EPSO "innovations" advertising available at http://ec.europa.eu/avservices/player/streaming.cfm?type=ebsplus&sid=176005. Whereas it is evident that having succeeded to reduce the previous 2-year process to 10 months is a positive advancement, as a recurrent participant to EPSO's tests and having experienced numerous of those "innovations" at different stages, I wonder how you can substantiate the claimed "success" of the 2010 General Competition if you deny candidates the right to transparency to review the questions they have sat. Dozens of candidates have been denied the essential right to review the validity of the questions despite requesting so to EPSO. Yet, as you may be able to witness directly by trying the CBT self-assessment that you have introduced in the registration process for the 2011 General Competition, you will find at least 3 erroneous questions out of 20 in verbal, numerical and abstract reasoning. Even if there was only 1 wrong question, as the system is set, it would still be statistically random and thus unacceptable.
You will certainly agree with me that whether the margin of error is 5% or more in the more likely order of 15%, it does not set out the basis for an objectively "successful" preselection of a 'top' 1 to 2%. With my best of intentions and concerns for our Institutions, before it has to be dealt with by the Courts who has increasing evidence of the floors, I would kindly call for your urgent acknowledgement of the margin of error by instructing an amendment of the CBT for the 2011 competitions accordingly. From a quantitative point of view, the challenge will be simpler for EPSO if the threshold preselection pass mark was of an approximate 80%, thus avoiding the unavoidable margin of error problem. Please see Annex 2 to my letter attached for further suggestions.
Kindly note that even if for 2011 you have introduced 40 additional questions, assuming 'Situational Judgement' is error-less (which is unrealistic), given the shear numbers of candidates to be expected, the margin of error will most probably remain higher than the share of candidates to be preselected. In addition, please note that Situational Judgement requires clear deontological or other parameters to actually 'situate' and contextualise the question. The examples already available in EPSO's website set out rather subjective and questionable conclusions. Allegedly, Situational Judgement works for medical practice or as a snapshot of the potential working attitudes of a candidate. But using it as a selection tool in which the top 1-2% have to be selected seems very unreasonable and more so to experiment it with thousands of candidates.
I can also confirm that the new reform has detracted many highly-qualified candidates including many of my colleagues. We do not know on any elite institution basing its selection exclusively on pretiction of performance. I am myself wondering if it is worth to try again despite last year I got 35/40 in the CBT. Simply the fact that I am convinced that there were at least 2 CBT questions wrongly formulated in my CBT and that my appeal to Mr Giles Guillard to have access to the test questions was ignored makes me not want to participate anymore. I informed Mr Guillard also about the statistical problem but it seems it has also been ignored. In addition, I pointed out at the numerous claims from candidates who even got 'Neutralised' questions in their marks sheets. I really wonder whether it is worth for me again to spend 1,000 Euros to travel back to attend statistically random tests even if my average CBT performance is in the range of 90%.
Please do not hesitate to contact me if you would like any additional feedback from an experienced witness of EPSO's tests. I would genuinely be more than happy to provide you with as much constructive feedback as possible and required.
Sincerely yours,
Jaime ROYO-OLID
Attaché Infrastructure & Rehabilitation
European Union Delegation to Eritrea
Po BOX 5710, Zone 2 Sub zone 03
Marsa Teklai 192 Street, house Nr 20/22
Asmara, Eritrea
Tel: +291-1-12.65.66 Fax: +291-1-12.65.78







From: epso-robomail@ec.europa.eu mailto:epso-robomail@ec.europa.eu
Sent: Monday, March 07, 2011 10:27 AM
To: jaime.royo-olid@ec.europa.eu
Subject: EPSO Reference: 20110307-C9VSF3ME


          • PLEASE DO NOT REPLY TO THIS MESSAGE! *****
Use our webform for further correspondence: http://europa.eu/epso/apply/contact/details/index_en.htm



Thank you for your e-mail. You will find a copy of the message you have sent to EPSO below.

You will receive a reply as soon as possible.

Your reference number is: 20110307-C9VSF3ME

Please use this number in all further correspondence with EPSO.

Your EPSO Candidate Contact Service

Your candidate Nº 1747961
Last name Royo-Olid?
First name Jaime
Gender Male
Email jaime.royo-olid@ec.europa.eu
Telephone (with international prefix) +2911126566
Reference nº of a previous mail 20110307-C9VSF3ME
Subject Ongoing competitions for permanent officials
Other
EPSO/AD/185/10
Write your message (maximum 3000 characters) Dear EPSO representative,

Following my participation at EPSO/AD/185/10 - ENGLISH-LANGUAGE (EN) TRANSLATORS, CBT, on December 1st 2010 I was informed that "the candidates who have been invited to attend an assessment centre session obtained a minimum total of 48.67 points in the admission tests". Could you please explain how EPSO established the 48.67 thresholds mark?

I would also appreciate if you could clarify if the verbal, numerical and abstract reasoning components of this CBT where set to equivalent difficulty levels to the EPSO/AD/177/10-EPA.

Sincerely yours,

Jaime Royo-Olid?






Part 7 - If the complaint concerns work relationships with the EU institutions and bodies: have you used all the possibilities for internal administrative requests and complaints provided for in the Staff Regulations? If so, have the time limits for replies by the institutions already expired?
Not applicable

Part 8 - Has the object of your complaint already been settled by a court or is it pending before a court?
No



My particular complaint brings about the new case that there is since 16 March 2011 publicly-accessible irrefutable evidence over the inherent margin of error of EPSO's CBT testing published in its webpage. This evidence proves that CBT has been and will continue to be statistically unfit to pre-select a smaller % of candidates than the % margin of error.

However, the more general case of EPSO objecting to grant candidates access to CBT tests was addressed by the Ombudsman own-initiative inquiry OI/4/2007/(ID)MHZ. In that case, the Ombudsman refers to a number of pending court cases which would be related but not the same issue.





Part 9 - Please select one of the following two options after having read the information in the box below:
Please treat my complaint publicly

Part 10 - Do you agree that your complaint may be passed on to another institution or body (European or national), if the European Ombudsman decides that he is not entitled to deal with it?
Yes

Created by: admin last modification: Saturday 02 of April, 2011 [13:39:08 UTC] by admin

wiki page: EPSO Laureate 01 · EPSO 03 · Manipulating?Internal?Competitions ? · X AD 229 11 EN · X AD 229 11 FR · Steps CA 01 · Has EPSO Failed · Same Job but at Half Salary · Manipulating Internal Competitions ? · Errora EPSO 01 · Errors EPSO 01 · Fastest Test in the West EPSO/AST/111/10 · No discrimination for Contractual agents means no frustration amongst them · AN DIE TEILNEHMERINNEN DES ALLGEMEINEN AUSWAHLVERFAHRENS EPSO/AD/177/10 VERWALTUNGSBEAMTE AD 5 · A TUTTI I PARTECIPANTI AL CONCORSO GENERALE EPSO/AD/177/10 AMMINISTRATORI AD 5 · TO ALL PARTICIPANTS OF GENERAL COMPETION EPSO/AD/177/10 — ADMINISTRATERS AD 5 · Contre la Discrimination dans le cadre des Concours Internes – 2-ème Pétition · A TOUS LES PARTICIPANTS AU CONCOURS GENERAL EPSO/AD/177/10 · EU2 internal competitions-whose interests do they serve · Complaint regarding general competition EPSO/AD/177/10 · A Day in the Life: Contract Agent in a Delegation · All you ever wanted to know about SID · What are the grievances of the Contractual Agents · La vérité sur les concours internes EPSO · Concours internes – toujours discriminatoires · Petition Against Discrimination in Internal Competitions · Les nouveaux Concours Internes EPSO Discriminant les Agents Contractuels · Zanizanie stopni funkcyjnych pracownikow Komisji · Degradando al personal de la Comision · EPSO Tests for Contractual Agents Sources for the Quantitative and Verbal Reasoning Test · Devaluation du personnel de la commission · Downgrading the Commission staff ·

Current events
Powered by Tikiwiki Powered by PHP Powered by Smarty Powered by ADOdb Made with CSS Powered by RDF
RSS Wiki RSS Maps rss Calendars
Powered by Tikiwiki CMS/Groupware | Installed by SimpleScripts