If we had "plenty of refineries," they wouldn't have just broken ground on the first new refinery in around 30 years...well that and your proximity to a major metropolitan area. LA, SF, Miami, NY, DC probably all have the highest average rates in the country. more cars in an area = more potential profit for oil companies. as there are more cars in CA than anywhere in the world, on average we get hit the hardest. and trust me there are PLENTY of refineries and a massive shipping/receiving industry out here.
BTW, gas costs more in different regions not only because of the distance from the refineries, but also because different regions have different mandated fuel blends (California is a prime example of this). All refineries are not the same, and do not produce the same fuel blends, so those mandates keep production capacity artificially low versus a high demand, and that keeps prices higher in many of those areas.
You might remember how fuel prices equalized across the country after Katrina hit back in '05--that was primarily because the EPA temporarily lifted the regional blend mandates to help offset the loss of refining capacity along the Gulf Coast.
Then again, $100+/bbl crude isn't helping matters any...