He was ordained by Beth Joseph Rabbinical Seminary of Brooklyn, New York and has a PhD in history from Harvard University. He served in the rabbinate, notably at the Riverdale Jewish Center in the 1960s, and as professor and chairman of the Department of Jewish Studies at CUNY (City University of New York). Together with Elie Wiesel, he founded CLAL: The National Jewish Center for Learning and Leadership, which offered pluralistic Jewish learning for Jewish communal leadership and programs of intrafaith dialogue for rabbis of every denominational background. He served as founding president of Jewish Life Network/Steinhardt Foundation, which created such programs as Birthright Israel and the Partnership for Excellence in Jewish Education. Rabbi Greenberg was one of the activist/founders of the Student Struggle for Soviet Jewry in the movement to liberate Russian Jewry and served as the chairman of the U.S. Holocaust Memorial Museum in Washington, D.C.
Rabbi Greenberg has written extensively on post-Holocaust Jewish religious thought, Jewish-Christian relations, pluralism, and the ethics of Jewish power.
He is currently president of the J.J. Greenberg Institute for the Advancement of Jewish Life and senior scholar-in-residence at Hadar.