He started his racing career as a car owner in California. One week his driver didn't show up, and he raced the car to a third place finish. He won a championship and several races in six years.
He attempted his first East Coast race at Darlington Raceway in 1953. Lee Petty invited Panch to race in the 1954 Darlington race, where he finished third. The finish impressed Tom Horbison, a Pennsylvania car dealer, who hired Panch to race his car during the 1955 season, often out qualifying and beating the works Ford cars. His 1955 finishes impressed Pete DePaolo, who hired Panch to race in his factory Ford team. Panch won his first Grand National race on July 20, 1956 at Montgomery, Alabama, taking pole and dominating the 100-mile event, finishing a lap ahead of second placed Buck Baker.
He won the first two races in 1957 at Lancaster, California and Concord, North Carolina. Took another victory at Spartanburg, South Carolina in April. But on June 6 when Billy Myers' Mercury ripped through a guardrail at Martinsville in May killing an eight year old boy the already anxious Automobile Manufacturers Association board decreed that the whole industry should take no part in "automobile races or other competitive events in which speed or horsepower are emphasized." And, furthermore, to "advertise or publicize ... capabilities of passenger cars for speed, or specific engine size, torque, horsepower or ability to accelerate or perform in any context that suggests speed" was forbidden.
So the "factory" teams of the day instantly became independents, which always meant they would operate with a lot less money, if at all. The Ford team, now run out of the Holman-Moody shop in Charlotte, gave each of it's drivers a couple of race cars to run the rest of the season. Panch was able to win three more races that year and finished second in the championship. Despite the six wins and 22 top-five finishes in his 42 starts that year, Panchs' career went backwards following the departure of the Ford factory team. In the next three seasons he only competed in 24 races.
It wasn't untill 1961 driving for "Smokey" Yunick that he got the right car again. He finished second in the second of two preliminary races leading up to the Daytona 500. Before the 500, Yunick told Panch to stay back from Roberts in case he blew an engine or crashed. Panch said, "I was suppose to stay about a half lap behind him (Roberts) and that's where I was when his engine uncorked." Roberts, who had won the pole for the 500, had totally dominated the race leading for 170 laps before his engine blew up with just 13 laps remaining. With Roberts out Panch got the order to go for it. He took the lead and finished 16 seconds in front of Joe Weatherly for his biggest win and the victory that provided a much needed boost to his career.
A little more than a year after his Daytona 500 win, Panch was hired by the Woods Brothers to drive their Ford. He made 69 starts for them between 1962-66, scoring 8 wins, 11 seconds and 11 third place finishes.
However on March 27, 1966 Ford Motor Co. pulled out because of a rules dispute with NASCAR president Bill France and for the second time in his career, Panch had lost a drive when Ford elected to withdraw from the sport.
Richard Howard, vice president and general manager of the Charlotte Motor Speedway was looking for drivers for a race and call Panch. Panch said he didn't have a car and Howard suggested that he called Lee Petty. He did and put a deal together that resultled in him winning the Charlotte 600 for his 17th and last career victory. Panch raced for Petty Enterprises until he announced his retirement after the National 500 at Charlotte in October 1966.
His wife Betty founded the Winston Cup Racing Wives Association.