Goodman described his childhood as alone and withdrawn after his father had died so early and his brother had left to go to college. He was bullied at school for being overweight. Goodman was in the boy scouts until 9th grade, which he said offered him the structure and camaraderie he missed; he additionally looked to Boy Scout leaders, and later, acting coaches, as father figures. His brother later returned home to help raise Goodman and his sister. As a child, Goodman spent a lot of time listening to the radio and reading comic books, initially subscribing to DC's Green Lantern and The Atom, and later turning to Marvel Comics. He also read his brother's copies of Mad and would later confess to shoplifting its paperback editions. Goodman credits his brother with introducing him to comedy and bebop.