How do you find out what families owned Slaves when Slavery was Legal in the USA?

Just want to know where my Ancestors lived. like do they have a list somewhere of the Slave Owners? I'm sure the government must have kept records of slaves since they were like property and taxable