Klein was born in New York City into an impoverished Jewish family.^[5] He graduated from high school early and enrolled at the City College of New York at the age of 14^[5] to study sociology. He joined the US Army and was stationed in Germany and later France, where he would permanently settle after being discharged.

In 1948, Klein enrolled at the Sorbonne, and later studied with Fernand Léger. At the time, Klein was interested in abstract painting and sculpture. In 1952, he had two successful solo exhibitions in Milan and began a collaboration with the architect Angelo Mangiarotti.^[5] Klein also experimented with kinetic art, and it was at an exhibition of his kinetic sculptures that he met Alexander Liberman, the art director for Vogue.