Oil-filled Capacitors
Oil-filled capacitors have been used for many years in a variety of high-current and/or high-voltage applications, like motor-run, energy storage, and power factor correction.  The oil helps cool the large capacitors, it displaces air to help prevent corrosion and reduce the chance of destructive corona.  It increases the voltage breakdown for a given dielectric thickness.  The term "oil" is used here to mean a wide variety of liquids, many of which you might not think of as being "oils".  For that reason, some people prefer the term "liquid filled" instead of oil filled.  The term "mineral oil" is loosely used to mean just about any non-chlorinated liquid, including refined petroleum oils, silicones, and synthetic esters. 
Oil Filled
Very early high-voltage capacitors used things like petroleum oils and waxes.  They were made with mineral oil-impregnated paper as the dielectric, and aluminum foil electrodes and were first used in the 1920s.  The paper now used is a strong, long-fiber, all wood-pulp type called "kraft" paper (kraft means "strong" in German).  Linen based paper, used until the early 1940s, was found to have a shorter life. 

In 1927, a new class of "oils", the polychlorinated biphenyls (PCB), was introduced to industry.  Swann Chemical was the first US manufacturer, but Monsanto bought their business and became the sole US manufacturer.  PCBs are a group of 209 compounds in which biphenyl has been modified with chlorine.  About half have been used commercially at one time or another.   PCBs are normally used in mixtures, either with just each other, or with other chlorinated, and non-chlorinated compounds.  Pure PCBs are often heavy liquids, or even solids at room temperature, but the mixtures can have more fine-tuned physical properties.  The various PCB mixtures were known by the generic name of  "askarels" (actually a Monsanto trade name) and Aroclor was a dominant trader name for "refined" PCBs, a mix of closely related compounds.  Aroclors are often identified by a number with the format A-12 XX.  The A-12 means a refined Aroclor and the XX means the average % of Chlorine by weight.  So, A-1268, meaning an Aroclor with an average of 68% chlorine.  Blends of PCBs and other chemicals had a different naming system.  PCBs were sold under many trade names, like Aroclor (Monsanto), Pyroclor, Pyranol (GE), Dykanol (Cornell Dubilier), Noflamol, Inerteen (Westinghouse), and Hyvol.  I have seen estimates of 80 to 100 names in total.  The US used about 75% of the worlds PCB, about 1/3 of that imported.  They were used in a wide variety of industrial processes.  An important one being dielectric fluids in paper-in-oil capacitors.  They were also used as plasticizes, coolants, and as additives in lubricants, inks, paint, and adhesives, to name a very few applications.  Capacitors were the major use for PCBs, they may have accounted for 50% of the total.  Big transformers were the next biggest users at about 25%, where they acted as cooling fluids.  Other chlorinated compounds were sometimes used as well.  Halowax for example, a family of polychlorinated napthalenes.  They range in viscosity from light oils to hard waxes. 

PCBs were a big hit in the electrical industry; they had reasonably good electrical properties, very good high-temperature stability, high flash points, good resistance to corrosive chemicals, and low moisture absorption (moisture drives down the breakdown voltage).  Their use in big transformers was actually required by building codes in some areas.  One shortcoming was cost.  The much higher cost of PCBs over mineral oils forced changes in capacitor construction to reduce the amount of oil used for a given kVA rating.  A more fundamental problem was the very concept of the classic paper-oil-foil design.  Oils have much higher dissipation factors than more modern dielectrics, and the paper acts as insulation to help hold in the heat.  These problems limited high-power capacitors to about 100 kVA.  Big power distribution capacitors were not the only ones to use PCBs.  Many smaller capacitors were used in all kinds of electrical equipment such as motor run, lighting circuits, and many small home appliances like microwaves and TV sets.

In the 1930s, PCBs became associated with a variety of health risks, but it took until 1977 that their use and manufacture was either banned or severely restricted in the major of industrialized countries.  It is reported that Monsanto fought hard to defend a profitable business from government regulation, partly by limiting applications to closed systems unlikely to involve spills.  However, although the U.S. and Canada stopped production in 1977, world PCB production remained significant if falling.  The Soviet Union was still making PCBs as least as late as 1990.  In 1988, total production to that time was estimated at 1.2 million tons.  About one third was already in the environment, two thirds in storage or in use, and only a few percent having been destroyed.

PCBs are believed to cause liver and nerve damage, are believed to cause cancer, and to be teratogens (cause reproductive damage).  Furthermore, their good chemical stability means that they biodegrade extremely slowly.   PCBs are fat soluble rather than water soluble and are not quickly eliminated by the body.  This allows disproportionately high levels of accumulation in body fat from low level exposure (called bioaccumulation).  They are also concentrated by the food chain, sometimes many thousands of times (called biomagnification).  Under partial oxidation they produce polychlorinated dibenzofurans, far more dangerous than the PCBs themselves.  It may be that PCDs (and similar products of PCBs) are what have given PCBs much of their bad reputation. 

PCBs and their many cousins present several types of exposure risks.  The first is contact during handling of the oil, manufacturing of the capacitors, and leakage of the capacitors in use.  The second is the risk of fire.  There are a number of cases in which electrical fires resulted in the contamination of entire buildings with PCBs and their combustion products, such as chlorinated dibensofurans and dioxin.  Decontamination after such fires has been extremely expensive, in some cases exceeding the original cost of the building.  These fires are also a terrible risk for fire fighters.  A third risk is from PCBs entering the environment and into our food chain, through accidental (in some cases deliberate) spills, or disposal in unsecured landfills.

Large PCB-filled transformers and capacitors have mostly been destroyed or decontaminated because the risk of fire or leakage is considered to be too great to leave them in service.  However, small PCB capacitors (such as motor-starter size) may still be used in industrial equipment if all the proper paperwork is filed.  Penalties for failure to follow the rules can be severe.  By Federal law, all non-PCB liquid-filled caps must be marked as such.  If a capacitor is not so marked, it must be assumed to contain PCBs.  Some small PCB capacitors were in the distribution pipeline long after manufacture stopped, and it may still be possible to purchase one by accident.  The EPA defines small as less than 3 lbs of PCBs.  There is still a significant number of PCB capacitors in old consumer products.  Submersible pumps for water wells are a special problem.  They have a long life so there are probably lots of them still in service.  Plus, if the capacitor leaks, the PCBs can go directly into drinking water.  Capacitors containing PCBs are easily found on-line. 

PCBs can be destroyed by high-temperature incineration; the chlorine is released as hydrogen chloride.  Mineral oils contaminated by PCBs (as from transformers) can be cleaned by chemical means, and a number of companies do this.  Biodegradation is possible is possible but slow.  Most bacterial that work well in the lab don't do as well in the wild. 

The banning of PCBs started the search for satisfactory substitutes.  Since then, a variety of mineral oils have been used in capacitors and other electrical applications, and research continues today.  The various fluids used include high-temperature hydrocarbons (like diaryl ethane), silicones, fluorocarbons, and synthetic esters.  Polypropylene glycol, polybutene, dioctylphthalate, diphenylethylane, and polyethylene glycols and related compounds are sometimes used, as are modified soybean oil, castor oil and rapeseed oil.  Some chlorinated hydrocarbons with good biodegradation and photodegradation properties are used as well.  They have names like Faradol and Dielectral.  Oil plus wax is sometimes used, as is epoxy.  Inert gas can be used to prevent corrosion.  Not all the replacement fluids are as nontoxic as might be wished, and some of them are now under attack as possible health hazards.  There is fear that some of these compounds may present future disposal problems.  At least they are more biodegradable than PCBs.

There been a "long march" of improvements in high-power capacitors.  In the 1963, polypropylene (PP) was introduced as a dielectric by G.E., and was used in PP/oil/paper hybrid designs.  PP has a much lower dissipation factor than oil, and a higher voltage breakdown.  PP has a dissipation factor of less than 0.1%, while the various oils are typically 0.5 to 3.5%, depending on type and temperature.  Oil and paper were still used, as a way to eliminate air pockets that would shorten the life of the capacitors.  PP allowed for construction of capacitors with much higher kVA ratings.    There has even been significant improvement in the quality of kraft paper, and its additives.  The another major step was to eliminate the aluminum foil in favor a vacuum metallized layer directly on the PP film.  Metallized PP film can self-heal, but also has a higher ESR so it is not used in all applications.   Manufacturers often replace the aluminum film with aluminum/zinc in various ratios for better self-healing.  Paper may still be used to hold the oil because oil does not wet the PP film very well.  Roughening the film allows it to hold an even film of oil without the paper however.  This is called "Hazy film", introduced in 1979 by G.E., and Hazy is a G.E. trademark.  Capacitors that use the traditional paper/oil layer are sometimes called "soggy film" capacitors.  All of these construction methods have their strengths and weaknesses, and all are still in use. 

Some people claim that PCB´s dangers are exaggerated and largely unproven (ha), and there has been some call to reintroduce them into certain applications.  Whatever the case, the levels of PCBs in the environment and in ourselves is reported to be on the decline. 

PCBs were not the only chlorinated compounds to come under attack.  PCBs are only a part of the much wider problem of the "persistent organochlorines".  Others include chlorinated phthalates, furans, paraffins, terphenyls, naphthalenes, plus a variety of herbicides, insecticides, nematicides, and fungicides, to name but a few.  Don't get me started on bromine.  What makes these compounds so dangerous (and so useful in many cases) is that they are often very similar to compounds found in living organisms, and therefore, can easily interact with these organisms.

But so what?   Bisphenol A, parabens and triclosans (biocides), phalates, alkyphenols, glycol ethers, ammonium perchlorate, cyclosilanes and many other (100s, perhaps 1000s) of chemicals known or suspected to be endocrine disrupters, asthma inducing, or have other affects on human health have been dumped into the environment with insane abandon.  PCBs are just one class of bad chemicals among many. 

Newer problem chemicals: 

di (2-ethylhexyl) phthalate (DEHP) was used as a replacement for PCBs in things like ballast capacitors from 1979 through about 1991.  DEHP has many of the same problems as PCBs and has been replaced.  

For more information see:
GE capacitors using Pyranol.  Taken 1955. 
http://www.bvws.org.uk/405alive/tech/pcbs.html  List of common trade names of PCBs. 
http://en.m.wikipedia.org/wiki/Polychlorinated_biphenyl  Good information and many references. 
http://www.deq.state.or.us/lq/cu/nwr/PortlandHarbor/docs/SourcePCBs.pdf  Many references. 
http://cdn.eastchem.com/therminol/Applications_of_PCBs_Erickson_Kaley_Aug2010_AuthorsProof_0.pdf  Very authoritative.