Born at the family home of “Collicella,” in Harrisonburg, Algernon Sidney Gray was the eldest son of Robert Gray and Isabella L. Waterman. Gray attended Washington College (now Washington and Lee University), Lexington, Virginia and opened his practice as an attorney in his hometown.

As one of Rockingham County’s delegates to the Virginia secession convention (John Francis Lewis and Samuel Augustus Coffman being the other delegates from Rockingham County), Gray opposed secession and was particularly remembered for a rather emotionally charged speech that brought the delegation to tears. Nonetheless, Virginia opted for secession and Gray reluctantly accepted the decision of the majority. Though he took little part in advancing the cause of the Confederacy, he was known for his support for those who suffered and were in distress, no matter their particular sentiments. He served as a member of the Board of Visitors for Washington College from 1861-1862.

After the war, Gray, as a member of the Republican Party, served in the Virginia Senate (1865–67). Beyond the reconstruction years, Gray continued his practice as a lawyer, helped develop mineral resources in Virginia, and, in 1871, was appointed the U.S. Marshal for the Western District of Virginia.

He died on 29 September 1878, at the age of 63, and was interred in Woodbine Cemetery, Harrisonburg.