When we visited the US PAYG sims were theoretically available but no one actually had any. The ATT website does not feature PAYG sims but you can get a refurb with a PAYG package for $10.

We ended up buying PAYG phones in Walmart and swopping the SIMs into our phones. This worked fine for us - the phones may be locked to the SIM but the SIMs were not locked to only work with that phone. YMMV