Sirtuin 5 (SIRT5) is a protein lysine deacylase enzyme that regulates diverse biology by hydrolyzing ϵ-N-carboxyacyllysine posttranslational modifications in the cell. Inhibition of SIRT5 has been linked to potential treatment of several cancers but potent compounds with activity in cells have been lacking. Here we developed mechanism-based inhibitors that incorporate isosteres of a carboxylic acid residue that is important for high-affinity binding to the enzyme active site. By masking of the tetrazole moiety of the most potent candidate from our initial SAR study, we achieved potent and cytoselective growth inhibition for the treatment of SIRT5-dependent leukemic cancer cell lines in culture. Thus, we provide an efficient, cellularly active small molecule that targets SIRT5, which can help elucidate its function and potential as a future drug target. This work shows that masked isosteres of carboxylic acids are viable chemical motifs for the development of inhibitors that target mitochondrial enzymes, which may have applications beyond the sirtuin field.