A survey found more than half of drivers have either reduced their hours or quit in response to rising fuel costs.