Political Ambition
During the May election of 1954, Garrett backed Lee Porter for attorney general with the intention of using Porter to retain political power. Albert Patterson, running on the platform "a man against crime," beat Porter in the primary election. Gene Wortsman and Edwin Strickland describe in their book, Phenix City, The Wickedest City in America, how Garrett and Arch Ferrell changed the numbers on a recapitulation sheet to add votes for Porter, convincing county democratic party chairman Lamar Reid that Patterson was the gamblers' candidate. Reid provided access to the sheets, with Garrett reassuring him later that "this vote thing comes up all the time."

Caught
When a Birmingham reporter found a discrepancy in the vote totals, Garrett was called before the Jefferson County grand jury to testify on June 18. Reid had confessed, naming Garrett and Ferrell in his testimony. After an all day session in front of the grand jury, Garrett returned to the Redmont Hotel.

Alibi
At the Redmont Hotel in Birmingham Garrett had dinner with with other Democratic Party workers and their wives. Several calls were placed between Ferrell's Phenix City office and Garrett's hotel room during that time. When Patterson was murdered that same evening, Ferrell used the phone calls to Garrett as an alibi.

Murder Acquittal
Si Garrett was called a second time to testify before the grand jury. This time he was questioned vote fraud and Patterson's murder. After ten and a half hours of testifying, Garrett left Alabama and checked into a Texas mental hospital.

In August, Garrett suffered a broken neck in an auto accident. He survived, but returned to the mental hospital. Garret was never brought to trial for either the vote fraud or the murder of Albert Patterson.