We're fine-tuning for clarity

Thomson Reuters last week launched a major project to create Global Institutional Profiles. The target is a rounded picture of what a higher education institution does and how well it does it.

There are many reasons for evaluating what any organisation does - how it "performs" if you like - but the "performance" issue boils down to a simple question: "is it any good?" That turns out to be rather tricky to answer, not least because it begs the rejoinder, "compared with what?"

The London School of Economics is generally agreed to be an outstanding institution globally. But how can we judge that? A lot of people would like to study there. If you wanted an informed opinion, you would consult the people who work there. A lot of people who have been there have gone on to great things. These are good indicators that the place is intellectually vibrant and delivers excellent teaching, and those values are endorsed internationally.

Good, but not perfect. Three major problems spring to mind. First, that quick summary tells us there are many ways in which we may value what a university does. It is a knowledge business and a source for teaching, research and dissemination to users. Second, the LSE is a specialist. Its astronomy is weak, so we need to consider subject portfolio. And, third, what will we measure? I need an informed expert to confirm my judgment, but as I can't send my expert to every institution, I need a proxy indicator (not a "metric": an indicator).

Our view of the LSE does not translate readily into anything useful unless we are careful and we make sure our information is appropriate. The LSE stood at only 67th in the last Times Higher Education-QS World University Rankings - some mistake surely? Yes, and quite a big one. LSE academics publish papers in social and economic sciences, which have lower citation rates than the natural sciences; so on the simple "citations per paper" used by QS in analysing the Scopus publications data, it slipped way down the list. Not a good way of comparing it with nearby King's College London, which has a huge medical school.

We need a lot more information than has typically been gathered before we can build an even halfway sensible picture of what a university is doing.

The Global Institutional Profiles project will address different areas of each institution's activity. Based on advice from experts around the world, we have created a series of data-gathering strands that Times Higher Education is helping us to pull together to make a much richer picture.

We will ask institutions to supply us with information about their students, staff and income for teaching and research. We will ask others to tell us what they think of institutions they know about, not just in terms of whether they are "good" but whether they value their graduates or use their research. And we will use our own data on their publications and their patents and how often these are used by other people.

We will also ask about activity at a faculty level where possible. We will not treat science, engineering, social science and humanities as if they were the same thing. We know they are not.

This database will first be used as the source of critical information to enable Times Higher Education to publish the most definitive set of World University Rankings so far. Ranking tables? Yes, but with more input variables, more transparent methodology and many additional elements to provide the nuances that explain the outcomes and reveal what diversity lies within.

Thomson Reuters will also be able to provide reports to institutions on their activity "in the round", allowing them - critically - to compare themselves with peers rather than global averages. This converts data into truly useful management information.

We know this enterprise asks the universities to work with us to get the right result. That is why we started by asking for your opinions on what we should be doing. Our start in 2010 will be the first, not the final, step towards developing a database for THE's rankings and our own profiles that are valued by university managers, by researchers and by students globally.