A report from the four bodies covering police, prisons and care called for a rethink of how powers are used to detain people in a "place of safety".
Some of those who were detained were as young as 14.
Current guidance says police should take the mentally ill to a hospital or similar location in all but exceptional circumstances.
But the report from Her Majesty's Inspectorates of Constabulary and Prisons, the Care Quality Commission and the Healthcare Inspectorate Wales said detention in police cells was far from exceptional.
Under section 136 of the Mental Health Act 1983, police officers can hold someone for their own protection and so that they can be medically assessed.
Examples include officers called to deal with an elderly person with dementia walking outside alone, people who may be suicidal, or incidents in which someone is suffering from a psychotic episode leading to abnormal or dangerous behaviour.
During 2011-12, 9,000 people were detained in police custody under section 136, while 16,000 were taken to a hospital.
This is "clearly not an exceptional use of the power", the inspectors said.
When they looked closely at a sample of 70 cases, they found that eight out of 10 of those held in police cells had been detained in relation to fears either that they were suicidal or that they could harm themselves.
The report said that people detained in police custody under section 136 were subject to the same "processes and procedures" as arrested criminals.
The average stay in a cell was more than 10 hours. Inspectors said the legal disparity needed to be resolved between mental health detainees, who could be held for 72 hours, and most criminals, who must be charged or released after 24.
The most common reasons for police custody were that there was neither a bed nor staff available at a hospital or other healthcare facility.
"It needs to be made abundantly clear that police officers do not wish to detain people under the mental health act but are all too often forced to so do due to a lack of provision in other services”
Other detainees were taken to cells because they were drunk, behaving violently, or known to have done so in the past.
The report said many police officers had told inspectors they did not think that custody was the best place for these people.
Drusilla Sharpling, of HM Inspectorate of Constabulary, said: "This report finds that too many people are being detained in police custody under section 136.
"Their only 'crime' is that they have mental disorders, but they are treated in many ways as if they are criminals. This deplorable situation cannot be allowed to continue."
The Association of Police and Crime Commissioners said: "It is a police officer's role to ensure the safety of the public and individuals who pose a threat to themselves or others.
What are your views? Is it the fault of the Police, or the Government for not providing the services for mental health patients for the Police to take them to?