A half-year visit to a US university under a J1 visa requires US health insurance. In the University of Pennsylvania, the website of the International Students’ and Scholars’ Office says that they have pre-selected some health insurance plans. According to their cover description and comparison websites, these plans offer significantly less cover than similarly-priced plans not pre-selected by U Penn. My spreadsheet comparing the health insurance options is public. Possibly I have missed some aspects of insurance that justify the price difference. Another explanation is that the pre-selection was done by people who do not themselves use these insurance plans (because they are not visitors to the US) and have little incentive to make an effort to choose the best plan. A more negative and less likely interpretation is that the insurance company is providing incentives (such as kickbacks) for the selectors at U Penn to direct visitors to expensive low-cover plans that are profitable for the insurer.
Suppose that a university employs roughly equal numbers of men and women overall, but the proportions of the genders differ across research fields, or across research and administration, and the university wants gender equality within each subfield. Using the narrowest definition of a research field, each field has either one or zero people, so in the smallest fields, gender equality is impossible. From now on, the focus is on fields or administrative units that have at least two people.
Gender equity can be achieved on paper by redrawing the administrative boundaries or redefining research fields. A simple algorithm (not the only possible algorithm) is to form pairs of one male and one female employee and call each such pair an administrative unit. Larger units can be formed by adding many male-female pairs together. If the numbers of men and women are not exactly equal, then some single-gender pairs will be left over, but most administrative units will have perfect gender equality.
The same idea can be used less radically by reassigning people who do interdisciplinary work and could plausibly belong to multiple research fields. Each person who is „between” fields gets assigned to the field that has a smaller fraction of that person’s gender. This increases gender equality in both the field that the person joins and the field that the person left. The field with a bigger surplus of that person’s gender loses one of that gender, and the field with a smaller surplus gains one.
Other than reassigning people or redrawing administrative boundaries, gender equity requires inducing people to change their research fields. To some of my colleagues, directing researchers to change their area is ideologically unacceptable. However, if equity is the goal and people’s field change is necessary to achieve it, then several methods can be used. The inducements can be softer or harder. A hard inducement is a hiring policy (and job ads) that restrict hiring to only one gender. The same can be done with promotion and retention. If this policy was explicitly stated to everyone, then it would become more effective, and also more acceptable to people than when they are surprised with it upon applying for promotion.
Soft inducements consist of hints that one gender is preferred, which are usually stated in political doublespeak like „we are committed to equity” or „we are an equal opportunity employer”. If many more candidates of one gender apply, then giving all candidates an equal opportunity of getting hired does not result in equal proportions of men and women employed. I am in favour of clear guidelines and transparency, for example of explicitly stating in job ads and promotion policies that the underrepresented gender is preferred, and which gender is currently underrepresented. Clearly telling people that switching fields is good for their career is likely to have a bigger effect than the currently used hints.
It may be easier to shift the research areas of people who are earlier in their careers. Encouraging more young people of a given gender to go to an area where their gender is underrepresented is one way of inducing them to change their field (relative to their preference).
Current equity policies are focussing almost exclusively on the inflow of employees. Gender balance can also be improved by managing the outflow, for example by offering early retirement schemes to one gender, or expanding these to a wider age range for one gender. If there are gender differences in the propensity of accepting certain inducements to leave, then the same inducements can be offered to both genders (seemingly gender-neutrally), with the desired result that one gender exits more.
Senior academics tell juniors that an assistant professor does not have to get tenure at his or her current university, but “in the profession”, i.e. at some university. To extend this reasoning, one does not have to get tenure at all, just guarantee one’s ability to pay one’s living costs with as low effort as possible. Government jobs are also secure – not quite tenure, but close.
Economically, tenure is guaranteed income for life (or until a mandatory retirement age) in exchange for teaching and administrative work. The income may vary somewhat, based on research and teaching success, but there is some lower bound on salary. Many nontenured academics are obsessed about getting tenure. The main reason is probably not the prestige of being called Professor, but the income security. People with families seem especially risk averse and motivated to secure their job.
Guaranteed income can be obtained by other means than tenure, e.g. by saving enough to live off the interest and dividends (becoming a rentier). Accumulating such savings is better than tenure, because there is no teaching and administration requirement. If one wishes, one can always teach for free. Similarly, research can be done in one’s free time. If expensive equipment is needed for the research, then one can pay a university or other institution for access to it. The payment may be in labour (becoming an unpaid research assistant). Becoming financially independent therefore means giving oneself more than tenure. Not many academics seem to have noticed this option, because they choose a wasteful consumerist lifestyle and do not plan their finances.
Given the scarcity of tenure-track jobs in many fields, choosing the highest-paying private-sector position (to accumulate savings), may be a quicker and more certain path to the economic equivalent of tenure than completing sequential postdocs. The option of an industry job seems risky to graduate students, because unlike in academia, one can get fired. However, the chance of layoffs should be compared to failing to get a second postdoc at an institution of the same or higher prestige. When one industry job ends, there are others. Like in academia, moving downward is easier than up.
To properly compare the prospects in academia and industry, one should look at the statistics, not listen to anecdotal tales of one’s acquaintances or the promises of recruiters. If one aspires to be a researcher, then one should base one’s life decisions on properly researched facts. It is surprising how many academics do not. The relevant statistics on the percentage of graduates or postdocs who get a tenure-track job or later tenure have been published for several fields (http://www.nature.com/ncb/journal/v12/n12/full/ncb1210-1123.html, http://www.education.uw.edu/cirge/wp-content/uploads/2012/11/so-you-want-to-become-a-professor.pdf, https://www.aeaweb.org/articles?id=10.1257/jep.28.3.205). The earnings in both higher education and various industries are published as part of national labour force statistics. Objective information on job security (frequency of firing) is harder to get, but administrative data from the Nordic countries has it.
Of course, earnings are not the whole story. If one has to live in an expensive city to get a high salary, then the disposable income may be lower than with a smaller salary in a cheaper location. Non-monetary aspects of the job matter, such as hazardous or hostile work environment, the hours and flexibility. Junior academics normally work much longer than the 40 hours per week standard in most jobs, but the highest-paid private-sector positions may require even more time and effort than academia. The hours may be more flexible in academia, other than the teaching times. The work is probably of the same low danger level. There is no reason to suppose the friendliness of the colleagues to differ.
Besides higher salary, a benefit of industry jobs is that they can be started earlier in life, before the 6 years in graduate school and a few more in postdoc positions. Starting early helps with savings accumulation, due to compound interest. Some people have become financially independent in their early thirties this way (see mrmoneymustache.com).
If one likes all aspects of an academic job (teaching, research and service), then it is reasonable to choose an academic career. If some aspects are not inherently rewarding, then one should consider the alternative scenario in which the hours spent on those aspects are spent on paid employment instead. The rewarding parts of the job are done in one’s free time. Does this alternative scenario yield a higher salary? The non-monetary parts of this scenario seem comparable to academia.
Tenure is becoming more difficult to get, as evidenced by the lengthening PhD duration, the increasing average number of postdocs people do before getting tenure, and by the lengthening tenure clocks (9 years at Carnegie Mellon vs the standard 6). Senior academics (who have guarateed jobs) benefit from increased competition among junior academics, because then the juniors will do more work for the seniors for less money. So the senior academics have an incentive to lure young people into academia (to work in their labs as students and postdocs), even if this is not in the young people’s interest. The seniors do not fear competition from juniors, due to the aforementioned guaranteed jobs.
Graduate student and postdoc unions are lobbying universities and governments to give them more money. This has at best a limited impact, because in the end the jobs and salaries are determined by supply and demand. If the unions want to make current students and postdocs better off, then they should discourage new students from entering academia. If they want everyone to be better off, then they should encourage research-based decision-making by everyone. I do not mean presenting isolated facts that support their political agenda (like the unions do now), but promoting the use of the full set of labour force statistics available, asking people to think about their life goals and what jobs will help achieve those goals, and developing predictive models along the lines of “if you do a PhD in this field in this university, then your probable job and income at age 30, 40, etc is…”.
Let’s distinguish the knowledge from the degree first. The average skill requirement of jobs (measured in years of education for example) is rising over time, so people need more knowledge before entering the labour market. What is obsolete is the packaging of that knowledge into degrees and perhaps its teaching in universities.
The PhD takes six years on average (http://gsa.yale.edu/sites/default/files/Improving%20Graduate%20Education%20at%20Yale%20University.pdf) and during that time the student is guided by one or at most a few advisors. Working on the same topic on years is often necessary to become an expert, so unavoidable. But being tied to the same advisor is a throwback to the medieval guild system where the apprentice and journeyman work years for the master. It means seeing only one viewpoint or set of techniques. Most importantly, the topic of the thesis is limited to what the advisor is competent in (sometimes a laissez-faire advisor allows a dissertation on an unfamiliar subject, which is even worse – incompetent advising follows). Taking courses from other faculty in the same department or university broadens the horizons a bit, but there may be an institutional culture that introduces biases, or expertise in some fields may simply be missing from the university. Attending conferences again broadens the mind, but conferences are few and far between. Suggestions that run counter to the advisor’s views may be interpreted as wrong by a novice graduate student.
Ideally, a trainee researcher would be advised by the whole world’s scientific community, mostly but not exclusively by people in the same discipline. Electronic communication makes this easy. Many different viewpoints would be explained to the graduate student, interpersonal issues would be easier to resolve by changing advisors (no lock-in to one person who determines one’s career prospects). People who just use students as free labour without providing much in return would suddenly become lonely. The problem is moral hazard – if no specific person has responsibility for a student, indefinite postponement of advising effort may occur. Credit for useful advice would be spread between many people, which dilutes incentives. In short, advising is a public good.
Still, public goods are sometimes provided, despite the difficulty of explaining this with a rational agent model. People write free software, answer questions from strangers in forums, upload advice and instructions on many topics. This suggests some volunteer advisors would step forward under a shared responsibility system. The advisor pool may become more ideologically biased than now, because people who want to spread propaganda on their strong views have a greater incentive to volunteer advice. They do this on the internet, after all. Similar incentives for shrill prophets operate in universities, but if each faculty member is required to advise some students or if there is a cap on how many disciples one can take, there is less scope for indoctrinating the masses. Such restrictions can be imposed online to some extent. There could be a reputation mechanism among the advisors, so the crackpots are labelled as such. The larger pool of opinions may balance the biases.
The economies of scale in advising one student are reduced with sharing. A single advisor per student means that during most of the PhD program, the advisor is already familiar with the student’s work and only needs to read the new part each week. With many advisors, each would need to devote time to the same material. Some sharing of responsibilities (one reads the introduction, another the conclusion) is possible, but the interdependence of the parts of the research does not permit full splitting.
Another medieval aspect of the PhD is paying for the received teaching in labour, not money. Graduate students may be free from tuition and may even get a scholarship, but in return have to work as teaching assistants or do the advisor’s research in their lab. Less ethical help also occurs, such as reviewing papers the advisor is officially the referee of. Inefficiencies of a barter economy are introduced. Instead of paying for the program with money earned in the job the student is the most productive or happy in, the student is forced to work as a teaching assistant and essentially pay the difference between a fair market wage and the teaching assistant wage to the university. Further, the teaching work is restricted to the university of the PhD program, even if other universities need teachers more and offer higher wages. This gives the university market power and allows it to depress grad student salaries.
A doctoral program may lose money directly, in the sense that teaching the grad students is more expensive (due to small classes, advanced material, so more professor time per student) than their TA work. The fact that universities still keep the PhD programs suggests the existence of indirect benefits. One is reputation – attracting paying undergraduate and Master’s students. In some countries, an institution is not allowed to call itself a university if it does not teach at the doctoral level. Altruism by the higher education sector is possible, even if John Quiggin’s quote “never stand between a Vice-Chancellor and a bucket of money” suggests otherwise.
One utopic proposal is an online system where graduate students and advisors sign up and can talk over video calls, send emails etc. It keeps a record who communicated with whom and how much. Later, data on the academic achievement and job market performance of students can be added, so advisors can be rewarded for their students’ success. There may also be some popularity index, meaning students rate their advisors and vice versa. But in the end, an advisor’s contribution should matter more than popularity, so the latter is optional. Advisors may look at and rate each other’s advising sessions to limit the spread of bad advice. Students can collaborate and may decide to meet in person.
For experimental science, lab space can be rented by student cooperatives. Instruction in the use of equipment can be given via video. Classroom space can also be rented directly by groups of students if needed. The students may pay advisors. Some people may only advise conditional on payment. Students may teach other students (including TA jobs), whether for money or pro bono. The system would cut out the middlemen – university administrators – making education cheaper for society. Of course, in the lab and classroom rental business, other middlemen would appear and take their share.
Universities usually prefer that the same person both teaches and does research. There are some purely teaching or purely research-focussed positions, but these are a minority. Both teaching and research achievements (and service as well) are required for tenure. This runs counter to Adam Smith’s argument that division of labour raises overall productivity. One possible cause is an economy of scope (synergy), meaning that teaching helps with research, or research helps to teach. In my experience, there is no such synergy, except maybe in high-level doctoral courses that focus exclusively on recent research. Revising old and basic knowledge by teaching it does not help generate novel insights about recent discoveries. Complex research does not help explain introductory ideas simply and clearly to beginners.
Another explanation is that universities try to hide their cross-subsidy going from teaching to research. The government gives money mainly for teaching, and if teachers and researchers were different people, then it would be easy for the government to check how much money was spent on each group. If, however, the same person is engaged in both activities, then the university can claim that most of the person’s time is spent teaching, or that their research is really designed to improve their teaching. In reality, teaching may be a minor side job and most of the salary may be paid for the research. This is suggested by the weight of research in hiring and tenuring.
The income of universities mostly comes from teaching, so they try to reduce competition from non-university teachers and institutions. One way is to differentiate their product by claiming that university teaching is special, complicated and research-based, so must be done by PhD holders or at least graduate students. Then schoolteachers for example would be excluded from providing this service. Actually the material up to and including first year doctorate courses is textbook-based and thus cannot consist of very recent discoveries. With the help of a textbook, many people could teach it – research is not required, only knowing the material thoroughly. For example, an undergraduate with good teaching skills who was top of the class in a course could teach that course next semester. Teaching skill is not highly correlated with research skill. The advantage someone who recently learned the material has in teaching it is that they remember which parts were difficult. A person who has known something for a long time probably does not recall how they would have preferred it taught when they first learned it.
Researchers forget the basics over time, because they rarely use these – there are more advanced methods. The foundations are learned to facilitate later learning of intermediate knowledge, which in turn helps with more complicated things and so on up to research level. Similarly in sports, musical performance, sewing, the initial exercises for learners can be quite different from the activity that is the end goal. A sports coach is rarely an Olympic athlete at the same time, so why should a teacher be a researcher simultaneously?
Getting grants counts positively in an academic’s evaluation and results in promotions and raises. But grants are supposed to be inputs for research, not outputs. Other things equal, it should be preferable to get the same output with fewer inputs (more efficiently and cheaply). Given an academic’s publications and patents, the grants they received in order to create these outputs should count negatively in their evaluation. The university administration is interested in motivating grant-getting, because they tax the grants – take a fraction of each for themselves. The motivating is done by promotions and raises. Rewarding more use of inputs inflates the cost of research and diverts effort from scientific output to getting more inputs.
A justification for rewarding grant-getting is that having current grants makes it easier to do research, thus increasing the expected scientific output in the near future. This only applies to a person’s current grants, not those already spent. Perhaps the current grants may count positively in an evaluation, but the spent ones should still have negative weight.
Once the system is in place, there may be an additional incentive to follow it: signalling obedience to rules. If academics are expected to apply for grants, then the ones that publicly do not may be considered contrarian, which may have negative consequences.
A similar reasoning applies to researchers from rich and poor institutions. If university resources are used for the work, then the person from a rich institution had more inputs for their work. The same output from a scientist in a poor university should be a more favourable signal about them.
An analogous adjustment is done in US college applications when low socioeconomic status confers an advantage. The direction of the correction is right, but its appropriate size remains to be determined.
If a donation is an expression of gratitude to a university where one acquired great skills or had a good time, then why not target it more precisely? Why donate to the entire university or a particular department as opposed to the people making up the university? Some people probably contributed more than others to the excellent university experience. It would make sense to reward them more. The people who made the studies enjoyable or useful may be gone from the university, especially if they were coursemates, but the employees of universities also change jobs. Those who are gone do not benefit from a donation to the university. A gratitude-based donation should go directly to the people one wants to thank.
If a donation is for the purpose of advancing education and research, then the money should be targeted to where it does the most good. But the universities receiving the most donations are those who are already rich. It is difficult to measure the benefit to education or research that an additional unit of money generates in different universities, but diminishing marginal returns seem reasonable. In that case, do-good donations should go to the poorest regions of the world and the poorest universities.
The richest universities often spend money on fancy architecture, with stonecarvings on the outside of buildings and woodcarvings and paintings inside them. The money thus spent clearly does not contribute to education or research. It may even have a negative value if architecturally interesting buildings are less well suited to study and work than a standard office block (this is true in my experience).
It is not enough to donate under the condition that the university must spend the money on scholarships or salaries, not buildings. There is a crowding-out effect: if the university receives a donation for a particular purpose, it spends less of its own money for that purpose than it would have without the donation. Effectively, part of the donation still goes to buildings.
The ranking of universities is very stable over time (http://m.chronicle.com/article/Rank-Delusions/189919) regardless of the difference in resources, scandals and other factors affecting popularity and quality. There are several positive feedback mechanisms keeping the rankings constant. These come from the multiple coordination games when choosing a university.
1) Smart and hardworking students want to be together with other smart and hard workers. If for some reason the best are in one location, then in the future all the best people have a motive to go to the same place. So the best arrive at that location and help attract other best people in the future. Similarly, party animals want to go to a university famous for its parties, and if many party animals come to a university, then it becomes famous for its parties.
Why would smart people want to be together with other intelligent folks? Just for interesting conversation, for useful contacts, collaboration. For these reasons, even the stupid may want to be together with the smart. Then an assortative matching results, as Gary Becker predicted for the marriage market (http://public.econ.duke.edu/~vjh3/e195S/readings/Becker_Assort_Mating.pdf).
2) Students want to go to a school with the best teaching staff, and the best professors want to teach the best students. I have yet to hear anyone wish for stupider students or teachers for themselves. Again the preference is the same among smarter and stupider students and professors, so assortative matching is a reasonable prediction.
3) The best professors want to be with other similar ones. Where there are such people, more will arrive.
4) Smarter graduates are likely to earn more and can donate more to the university. Then the university can hire better teaching staff, which in turn attracts better students, who donate more… The more talented also accumulate more power after graduating, in government institutions for example, which they can use (legally or not) to benefit their alma mater. Predicting this, again many people want to go there, and in the stiff competition the best get in.
5) If the employers believe that from some place, more intelligent people come than from elsewhere, then they are ready to make better offers to those coming from there. This makes the place attractive to all future employee candidates. Due to competition, the best get in, which justifies the belief of the employers.
6) Smarter people can be taught faster, at a pace that the stupider cannot keep. This mechanism gives everyone a motive to go to a school corresponding to their level.
7) Faster teaching means more knowledge during a standard length higher education, which the employers should value. The graduates of a school giving more knowledge are favoured, which makes the place attractive to everyone and leads to only the best getting in. The ability of the average student remains high, which enables faster teaching.
If only those who fail are allowed to retake an exam and it is not reported whether a grade comes from the first exam or a retake, then the failers get an advantage. They get a grade that is the maximum of two attempts, while others only get one attempt.
A simple example has two types of exam takers: H and L, with equal proportions in the population. The type may reflect talent or preparation for exam. There are three grades: A, B, C. The probabilities for each type to receive a certain grade from any given attempt of the exam are for H, Pr(A|H)=0.3, Pr(B|H)=0.6, Pr(C|H)=0.1 and for L, Pr(A|L)=0.2, Pr(B|L)=0.1, Pr(C|L)=0.7. The H type is more likely to get better grades, but there is noise in the grade.
After the retake of the exam, the probabilities for H to end up with each grade are Pr*(A|H)=0.33, Pr*(B|H)=0.66 and Pr*(C|H)=0.01. For L, Pr*(A|L)=0.34, Pr*(B|L)=0.17 and Pr*(C|L)=0.49. So the L type ends up with an A grade more frequently than H, due to retaking exams 70% of the time as opposed to H’s 10%.
If the observers of the grades are rational, they will infer by Bayes’ rule Pr(H|A)=33/67, Pr(H|B)=66/83 and Pr(H|C)=1/50.
It is probably to counter the advantage of retakers that some universities in the UK discount grades obtained from retaking exams (http://www.telegraph.co.uk/education/universityeducation/10236397/University-bias-against-A-level-resit-pupils.html). In the University of Queensland, those who fail a course can take a supplementary exam, but the grade is distinguished on the transcript from the grade obtained on first try. Also, the maximum grade possible from taking a supplementary exam is one step above failure – the three highest grades cannot be obtained.
Many universities advertise themselves as being among the top n in the world (or region, country etc). Many more than n in fact, for any n small enough (1000, 500, 100 etc). How can this be? There are many different rankings of universities, each university picks the ranking in which it is the highest and advertises that. So if there are 10 rankings, each with a different university as number one, then there are at least 10 universities claiming to be number one.
There are many reasonable ways to rank a scientist, a journal, a university or a department. For a scientist, one can count all research articles published, or only those in the last 10 or 5 years, or only those in the top 100 journals in the field, or any of the previous weighted by some measure of quality, etc. Quality can be the number of citations, or citations in the last 5 years or from papers in the top 50 journals or quality-weighted citations (for some measure of quality)…
What are the characteristics of a good ranking? Partly depends on what one cares about. If a fresh PhD student is looking for an advisor, it is good to have an influential person who can pull strings to get the student a job. Influence is positively related to total citations or quality-weighted publications, and older publications may be better known than newer. If a department is looking to hire a professor, they would like someone who is active in research, not resting on past glory. So the department looks at recent publications, not total lifetime ones. Or at least divides the number of publications by the number of years the author has been a researcher.
Partly the characteristics of a good ranking are objective. It is the quality-weighted publications that matter, not just total publications. Similarly for citations. Coauthored publications should matter less than solo-authored. The ranking should not be manipulable by splitting one’s paper into two or more, or merging several papers into one. It should not increase if two authors with solo papers agree to add each other as the author, or if two coauthors having two papers together agree to make both papers single-authored, one under each of their names. Therefore credit for coauthored papers should be split between authors so that the shares sum to one.
How to measure the quality of a publication? One never knows the true impact that a discovery will have over the infinite future. Only noisy signals about this can be observed. There currently is no better measure than the opinion of other scientists, but how to transform vague opinions into numerical rankings? The process seems to start with peer review.
Peer review is not a zero-one thing that a journal either has or not. There are a continuum of quality levels of it, from the top journals with very stringent requirements to middle ones where the referees only partly read or understand the paper, to fake journals that claim to have peer review but really don’t. There have been plenty of experiments where someone has submitted a clearly wrong or joke article to (ostensibly peer-reviewed) journals and got it accepted. Even top journals are not perfect, as evidenced by corrigenda published by authors and critical comments on published articles by other researchers. Even fake journals are unlikely to accept a paper where every word is “blah” – it would make their lack of review obvious and reduce revenue from other authors.
The rankings (of journals, researchers, universities) I have seen distinguish peer-reviewed journals from other publications in a zero-one way, not acknowledging the shades of grey between lack of review and competent review.
How to measure the quality of peer-review in a journal? One could look at the ranking of the researchers who are the reviewers and editors, but then how to rank the researchers? One could look at the quality-weighted citations per year to papers in the journal, but then what is the q uality of a citation?
Explicit measurement of the quality of peer-review is possible: each author submitting a paper is asked to introduce a hard-to-notice mistake into the paper deliberately, report that mistake to an independent database and the referees are asked to report all mistakes they find to the same database. The author can dispute claimed mistakes and some editor has to have final judgement. It is then easy to compare the quality of review across journals and reviewers by the fraction of introduced mistakes they find. This is the who-watches-the-watchmen situation studied in Rahman (2010) “But who will monitor the monitor?” (http://www.aeaweb.org/articles.php?doi=10.1257/aer.102.6.2767).
One could disregard the journal altogether and focus on quality-weighted citations of the papers, but there is useful information contained in the reputation of a journal. The question is measuring that reputation explicitly.
If there is no objective measure of paper quality (does the chemical process described in it work, the algorithm give a correct solution, the material have the claimed properties etc), then a ranking of papers must be based on people’s opinions. This is like voting. Each alternative to be voted on is a ranking of papers, or there may simply be voting for the best paper. Arrow’s impossibility theorem applies – it is not possible to establish an overall ranking of papers (that is Pareto efficient, independent of irrelevant alternatives, non-dictatorial) using people’s individual rankings.
Theorists have weakened independence of irrelevant alternatives (ranking of A and B does not depend on preferences about other options). If preferences are cardinal (utility values have meaning beyond their ordering), then some reformulations of Arrow’s criteria can be simultaneously satisfied and a cardinal ordering of papers may be derivable from individual orderings.
If the weight of a person’s vote on the ranking of papers or researchers depends on the rank this person or their papers get, then the preference aggregation becomes a fixed point problem even with truthful (nonstrategic) voting. (This is the website relevance ranking problem, addressed by Google’s PageRank and similar algorithms.) There may be multiple fixed points, i.e. multiple different rankings that weight the votes of individuals by their rank and derive their rank from everyone’s votes.
For example, A, B and C are voting on their ranking. Whoever is top-ranked by voting gets to be the dictator and determines everyone’s ranking. A, B, C would most prefer the ranking of themselves to be ABC, BCA and CAB respectively. Each of these rankings is a fixed point, because each person votes themselves as the dictator if they should become the dictator, and the dictator’s vote determines who becomes the dictator.
A fixed point may not exist: with voters A and B, if A thinks B should be the dictator and B thinks A should, and the dictator’s vote determines who becomes the dictator, then a contradiction results no matter whether A or B is the dictator.
If voting is strategic and more votes for you gives a greater weight to your vote, then the situation is the one studied in Acemoglu, Egorov, Sonin (2012) “Dynamics and stability of constitutions, coalitions, and clubs.” (http://www.aeaweb.org/articles.php?f=s&doi=10.1257/aer.102.4.1446). Again, multiple fixed points are possible, depending on the starting state.
Suppose now that the weights of votes are fixed in advance (they are not a fixed point of the voting process). An objective signal that ranks some alternatives, but not all, does not change the problem coming from Arrow’s impossibility theorem. An objective signal that gives some noisy information about the rank or quality of each alternative can be used to prevent strategic manipulation in voting, but does not change the outcome under truthful voting much (outcome is probably continuous in the amount of noise).