On May 13, the Higher Education Commission (HEC) issued a notification that with effect from fall of this year, universities are prohibited from conducting their own admissions tests for MS, MPhil and PhD programmes (graduate programmes) and that all applicants will now be required to take the Higher Education Aptitude Test (HAT) administered by the HEC’s accredited Education Testing Council (ETC).
The HAT was introduced as a lower-cost, domestic alternative to the globally recognised GRE and comes in a variety of types covering different subjects. This decree was issued right in the middle of admissions season in Pakistani universities, when some students may already have received admission decisions and are standing ready to start their graduate school journey in the Fall.
Just a few days later, on May 25, after blowback from universities, the implementation of the previous notification was postponed by a year, until Fall 2027. Clearly, this was a grand idea that sprang into a great mind at the HEC and was rolled out without consultation with any of the 250-or-so stakeholder universities affected by it.
But that is not the absurd part. What is absurd is that there is no evidence that this new indigenous aptitude test can do what it claims. Let me explain what I mean by that. The objective of aptitude tests such as SAT, GRE, LMAT, LSAT and MCAT is to predict students’ academic performance. For each of them, there is a large body of academic research and predictive validity studies spanning decades, conducted by entities developing such tests as well as independent researchers, that show the extent to which test performance is indicative of grades in the first year of the degree programs they were admitted to.
My question is: where are the predictive validity studies that demonstrate that student performance on the HAT successfully predicts academic performance in graduate school? I was easily able to find tons of studies for the GRE, but I could not find a single one for the HAT.
The successful design of tests with valid predictive value is not a matter of luck. It is the result of psychometric design. Psychometric design is the systematic development, testing, and validation of assessment instruments that measure latent traits of test takers with reliable precision and is a specialisation in the study of education. While introductory psychometric design is a part of most education programmes, developing expertise in it requires specialisation, practice and experience. In my 15 years working in Pakistan’s education sector, I have yet to encounter a critical mass of psychometric experts who are capable of populating question banks for the HAT. If the HEC or the ETC have somehow managed to corral such a group out of thin air, I am not aware of them, and I have not seen any evidence of their work.
I imagine that, just like the National Testing Service (NTS) did some 20 years earlier, somewhere in a sad room of the offices of the ETC, a few people have been handed test-prep guides for the SAT, TOEFL, GRE and other standardised tests bought from a local bookstore and given the task of plagiarising and entering questions into the HAT’s question bank.
If the questions in the sample tests posted on the ETC’s website are representative, then they have been filtered to keep only the easy ones. I estimate that any good grade-X student should be able to do well on such a test without any preparation. I am not convinced it holds much value for admissions to graduate school programs.
I also had the opportunity to speak with officials from a few of the country’s top universities. None of them can see any value that the HAT adds to their existing admissions process. Those who administer their own aptitude tests view the HAT as a clearly inferior option, which amounts to lowering their admissions standards.
So, if the psychometric expertise to develop standardised tests (for a wide range of disciplines) is lacking and there is no evidence that the HAT is able to meet its stated goals, what possessed the HEC to insert itself into the admissions processes of all universities and force them to adopt an unproven aptitude test that amounts to just another bureaucratic hurdle?
I believe the reason the HEC is foisting the HAT on students and universities as a mandatory requirement for admission is the same reason the government decided to go into the hemp cultivation business a few years ago – to make a quick buck. The cost for a student to take the HAT is Rs2,000. There are no official figures for the annual number of applicants to graduate programmes in Pakistan, but a guesstimate of 50,000 applicants would mean revenue of Rs100 million from HAT test fees. That is enough to justify at least a few cushy positions.
The HEC is concerned with a lot of tangential issues except for the ones that really matter at the end of the day: To what extent are students who graduate from degree programs successful in achieving their goals (find employment, secure admission to another programme of their choice, start a business, etc) within, say, a year of graduating? What is the value addition of degree programs? What are the real outcomes that graduates are concerned about? These outcomes are distinct from, and should not be confused with, the bureaucratic check-box exercise of tracking coverage of course and program learning outcomes that Pakistani universities have been chasing for the last decade in the name of Outcome-Based Education, which has done little to improve the substantive outcomes of student success that society cares about.
It all comes back to the same old question: what is the purpose of a university education? What is the goal of going through a degree programme? If it is to achieve the outcomes that the existing Outcome-Based Education framework tracks, then all is well, because all universities will be able to show you reams of paper demonstrating that their programs check all the boxes. But does that line up with students’ and society’s perception? Certainly not.
How is it that universities of the last two decades have vastly improved bibliometric measures quantifying so-called research outputs and generally risen in international rankings, yet we are grappling with some of the highest rates of youth unemployment among university graduates in the world (around 30 per cent)?
Who do our universities serve? Do they serve the people working in universities to meet their promotion requirements, publishing junk while relying on students serving as free labour or, at best, apprentices, or are universities meant to serve students in achieving personal and professional growth? At present, it is more the former, but it ought to be the latter.
Instead of devising half-witted money-making opportunities that interfere in universities’ admissions decisions and the perplexing pursuit of ‘uniformity’ without acknowledging the wide variation in types of higher education institutions and their applicant pools, it ought to track the level of substantive graduate success of programmes, because you cannot fix a problem unless you know your baseline, know where you stand, and can track which way things are moving, whether things are improving or deteriorating, and require programs that fall short to do better.
Until the HEC shifts its perspective from control to genuine advancement, this theatre of the absurd will continue to compromise the very foundations of higher education it was created to protect.
The writer (she/her) has a PhD in Education.
