Damian Green made the remarks to the Home Affairs Committee on 10 July after The Sunday Times and other newspapers reported that the prime minister was considering a change in policy.
The coalition government has pledged to reduce net migration to below 100,000 a year by 2015.
Critics of the government say that students who enter the UK from abroad should be counted only if they decide to settle permanently.
However, Mr Green said that it would be a “denial of reality” to say that “a human being who is here for three years is not an immigrant whereas the human being standing next to them who may have come on a work visa rather than a student visa is an immigrant”.
He argued that students, like other immigrants, use public services and that “trying to define our way out of the problem of uncontrolled immigration is wrong”.
Mr Green added that there were “long-term problems” in getting accurate information from the International Passenger Survey, which is currently used to estimate levels of migration. As a result, breaking down the data between students and non-students “probably couldn’t be done at the moment”.
Mr Green emphasised that there was no cap on the number of foreign students that universities could take, but to hit the 2015 target “we will need a reduction in student visas”.