Improvements to the exhaust system go hand in hand with improvements to the induction system. The exhaust system will consist of exhaust manifolds, interconnecting cross-over pipes for Vee engines, catalytic converters for pollution-controlled vehicles, mufflers and resonators, and the exhaust pipes. An easy way to gain a few horsepower is to install larger diameter pipes and mufflers when your old system rusts out. On a production line, parts are preformed so labour is a minor cost of installation. However, when replacing a rusted out system, the pipes are often formed by the installer so the materials are often a much smaller factor in the installation. In other words, it is not much more expensive to install a larger, free-flowing exhaust as it is for an original size, restrictive exhaust.
Often manufacturers make free flowing exhaust manifolds for cars equipped with performance engines. If there is room in your engine compartment for such manifolds, you should consider replacing them at the same time you are upgrading your pipes. The free flowing manifolds will likely be much larger than the restrictive ones and you will not be able to connect your existing pipes without some minor modifications by the muffler shop.
Exhaust headers are even less restrictive than manifolds and should also be considered. However, they have a number of disadvantages and should not be installed without due consideration. Firstly, they will make any engine compartment extremely tight and you may have a lot of trouble changing sparkplugs. They can make the engine compartment a lot hotter. They often don't fit perfectly and you may need to do a lot of work to make them fit. You may need to rejet your carburetor. Unless you get stainless steel, you will eventually have to replace them when they rust out. If you're still not scared, give them a try for a horsepower gain of up to about 7% at certain speeds.
Besides being smooth pathways to the exhaust pipe, the big improvement that headers have over manifolds is that they have a tuning capability. That is to say that they have a resonant frequency much like the pipes in a organ. When the engine's RPM reaches the resonant frequency of the headers, the pressure waves bouncing back and forth between the cylinder head and the header's collector will cause the exhaust flow to scavenge the maximum of amount of waste gases from the engine. This increases the engine's volumetric efficiency and results in a further increase in power at the resonant frequency. Ideally, the design RPM of the camshaft and intake manifold should match that of the headers.
The tuning effect of the headers is not quite as great as the potential gains on the intake side because 4-stroke SI engines are relatively insensitive to variations in exhaust manifold design as long as backpressure remains low. Unless you are building a race engine, headers will yield diminishing performance returns compared with the installation of a low restriction exhaust system.
While not performance related, you should also consider installing a stainless steel exhaust system if you have intentions of keeping your car for more than a few years. The stainless steel will pay for itself in about 3 years if you live in a northern climate and you drive your car year-round.