I also want to point out that the drive can take significantly longer than 5 hours. Google says it's a 5 hour drive, but I can't recall a time I was ever able to drive from the Bay Area to LA in that amount of time. There was always traffic either in the Bay or in LA or both. Do the math, it's almost impossible to find that perfect window during the day where you're not hitting rush hour somewhere. Granted, it depends on where you're leaving from. SFMM is on the northern end of the LA area, so it's possible to skip a lot of LA traffic if you're leaving from there.
Also, I highly recommend the other parks mentioned (especially SFDK). In fact, if you can only go to one Bay Area park, I'd definitely recommend SFDK or SCBB over CGA. Besides Railblazer (which isn't open to the public yet, so we shall see), I really don't think CGA is anything special. SFDK has a great B&M, most unique Impulse coaster, original Sky Rocket II, Joker RMC coaster, and the animals (dolphins!!). SCBB has been voted best seaside amusement park multiple times, and for good reason.
But really, it's completely up to you depending on your time and resources. Despite living in Northern California for most of my life, I never made the trip down to the SoCal parks and back in one or even two days. I pretty much devoted two whole days outside of my planned park days to just driving. Those are extra days you could be spending at Disney, the other parks, or just exploring LA. Bay Area is a whole different experience with a ton to offer that you could totally plan a different trip for.
Just things to consider.
“If toast always lands butter-side down, and cats always land on their feet, what happens if you strap toast on the back of a cat and drop it?”