Andrew Young, 40, suffered a head injury and died in hospital after the assault in Bournemouth in 2013.
Lewis Gill, of Sutton, south London, admitted manslaughter and was jailed at Salisbury Crown Court on Friday.
Justice Secretary Chris Grayling said the attorney general was considering seeking a longer sentence.
He called it a "repugnant crime", adding: "I think most of the public will feel justice hasn't been done."
"In a case where [the attorney general] judges the sentence to be too lenient, he can go back to the courts and seek a longer sentence," Mr Grayling said.
"He may choose to do this in that case."
Dorset Police described the killing outside a Tesco Express store in Charminster Road as a "violent attack on an innocent man".
CCTV showed Mr Young, who had Asperger's syndrome - a form of autism - apparently challenging Gill's friend, who was cycling on the pavement.
Moments later, Gill, 20, who was walking along behind, was shown punching Mr Young in the face.
Mr Young fell backwards and hit his head on the ground.
Conservative MP David Davies told the Daily Mail it was an "outrageously lean sentence".
Chris Grayling told Daily Politics the sentence was being reviewed
"In two years he will be out walking the streets after taking somebody's life," he said.
"He has attacked someone unprovoked and should be properly punished.
"People need to realise if you punch someone like that, and they fall backward, they can die."
Gill, who carried out the assault on 6 November, had an extra six months added to his sentence after admitting an unrelated charge of handling stolen goods and breaching a suspended sentence order.