Born on June 4, 1922 in Richmond Virginia, he attended Virginia Union University, leaving early to join the Naval reserves in 1942.
After basic training at the Naval Station Great Lakes Illinois, he entered V-12 Navy College Training at UCLA and then attended Midshipmen's school at Columbia University. He was commissioned as Ensign in 1944.
WWII was spent aboard the USS PC-1264, conducting patrols and escorts through the Atlantic.
In 1946, released from active duty, he returned to Virginia Union University and completed his Bachelor's in History, and worked as a railway postal clerk.
He returned to active duty in 1949, as a recruiter in Washington DC, transferring to the USS Iowa as a communications officer. He converted from the reserves to regular Navy in 1955.
In 1961, he was given his own boat, the USS Theodore E. Chandler, becoming the first African American to command a ship. His command transferred to the USS Falgout in 1962. During the Vietnam War he commanded the USS Taussig, and was the first African American to command a vessel during combat. He was promoted to Captain in 1967 - again being the first African American in the US Navy to do so, and in 1971, Rear admiral (also first).
He was given the command of the 3rd fleet based in Hawaii from 1978-1980 - the first African American to command a numbered naval fleet.
He retired in 1980, and spent his time fishing, traveling with his wife and breeding pigeons. He died on October 22, 2004 at Bethesda Maryland.
The ship USS Graverly was named in his honor.