Capacitors You Won't See Very Often.
Parylene:

Parylene is a group of versatile, very high-performance polymers most commonly used for board coating.  Parylene capacitors were manufactured by at least two companies but have disappeared.  Parylene is said to have properties similar to polystyrene but a much higher temperature rating.

Polyethylene: 

Polyethylene is hugely popular as a high-voltage cable insulation material in its cross-linked form, but I have never seen it used in commercial capacitors.  This is probably because of poor heat resistance (even lower than polystyrene).   It has some popularity with hobbyists doing high-voltage experiments however; the film is cheap and readily available.  Breakdown voltage is relatively high compared to other polymers.  Polyethylene has negligible moisture absorption, exceptionally low temperature drift, and very low dissipation factor.  I have seen some surplus mil-spec capacitors identified as polyethylene, but the standard they are built to, MIL-PRF-19978, does not actually specify which film is used, so polyester is more likely.  Other mentions of polyethylene have turned out to be polyethylene napthalate. 

Polyethylene is mentioned briefly in some handbooks in the 1950s, but there is no mention of the material after that.

Polyimide: 

Polyimide covers a number of related high-performance polymers that are used in a wide range of applications, Upilex, Kapton, Apical.  Some have been investigated as high-temperature capacitor dielectrics, but I don´t know of much usage beyond that.  Operating temperature for polyimides range from about 200-400C, depending on type.  High breakdown voltage and moderately low dissipation factor (as low as .3% for some grades).  Polyimides tend to be be fairly hydroscopic.  In any case, high cost and average DF might relegate polyimide to niche high-temperature applications.  Said to have poor self-healing.

Polyvinylidene fluoride: 

Polyvinylidene fluoride (PVDF, trade name Kynar) is rare as a capacitor material, I have never seen it in a stock capacitor but they are available.  Its high dielectric constant (around 8) has apparently made it useful in certain high-voltage pulse applications.  High-voltage storage capacitor in a defibrillation machine is one application.  Its temperature drift is good above 0C but very high below, and its dissipation factor is the highest of any plastic film.  Defibrillation capacitors are made as standard parts, but the dielectric is not stated.  More information would be welcome.

 



Polysulfone:
 

A few people spell it polysulphone.  Now gone, it was once considered to be the dielectric of the future.  The capacitor-grade film is probably out of production.  Very good heat resistance, to 150C.  Dissipation factor fairly good, and it remains good at relatively high temperature and high frequency.  Moisture absorption is high however.  Temperature drift is about the lowest among film capacitors.  Its major advantage is good high-temperature leakage.  I have never seen it in SMD. 

Polyethersulfone: 

Polyethersulfone is roughly similar to polysulfone.  I have seen hints that it has been used in capacitors.

Cellulose acetate: 

High dielectric constant (6 to 8) but a poor dissipation factor, poor temperature drift, poor leakage, poor dielectric absorption, and a limited temperature range, to 85C.  Siemens used this material to my knowledge, in the 1970s in a metallized form.

Wax and oil in paper: 

Wax and oil-impregnated paper (PIO) was once common for low-cost, small-value capacitors (100 pF-1 uF).  The original ones were built in wax-sealed cardboard tubes but later parts (1950s-'70s), were often made with plastic cases.  Performance and reliability was poor by modern standards and the operating life limited.  Humidity was a problem, making the parts prone to leakage and value change.  In damp climates, repairmen sometimes retrofitted TV sets with light bulbs to keep the insides warm and dry when the set was turned off.  Hobbyists rehabilitating old radios and amplifiers will normally replace wax paper caps with modern polyester or polypropylene parts, but a few still prefer PIOs.  A very few companies still make a limited number PIO capacitors, both oil and wax, for amature audio people and electric guitar rebuilders.  

The impregnants included vegetable oil (caster oil and perhaps others), paraffin oil and wax, chlorinated oils and waxes (Askarels and others), silicone oil, polyester resin, and other materials.  None of these had good characteristics by modern standards and all were somewhat different, one to another. 

In the days of vacuum tubes, the shortcomings of paper-in-oil capacitors were probably not so important. 

Class 4 Ceramics: 

These are the barrier-layer and reduced-titanate ceramics.  These ceramics have low breakdown voltage, high leakage and all-around poor electrical properties.  High Ks made them useful at one time but modern multilayer ceramics have made them obsolete.

Compressed gas: 

For a time, high-voltage/high-current capacitors were made using compressed gas, generally air, nitrogen or carbon dioxide, at pressures from several hundred psi to several thousand.  They used large metal bodies and could weigh several hundred pounds. They were not hermetic like later glass-body vacuum capacitors and needed to topped off periodically.  Eventually, sulfur hexafloride, helium and argon came into use. 

Polystyrene-polyester: 

An odd ball dielectric mix that came and went in the early days of polymer capacitors.  Heat resistance a little higher than polystyrene alone and temperature drift exceptionally low.  There are other mixed dielectric capacitors, some made to imitate the characteristics of polystyrene. 

Liquids: 

Tesla received a patent in 1891 (464,667) for a capacitor that used only a liquid (oil for example) instead of the more obvious paper-in-oil construction.  There have been a few patents that referenced Tesla's patent over the years, but I have no idea if this design has had significant application.  I have seen a few small variable capacitors with a liquid dielectric.  The advantage I suppose is that such a capacitor would have great self-healing in high-voltage applications.  Tesla was very much into high-voltage after all.  The down side is that the oils are often somewhat hydroscopic and absorbed water will pull down the breakdown voltage. 
Obscure Caps
Even more obscure dielectrics materials.  List is not complete: 

Many of these materials were investigated for their use in high temperature capacitors.  The success of PPS, PEN and other technologies may have cooled off interest in more advanced films for commercial use.  Research in high temperature, high energy and high density dielectrics is largely driven by the military and aerospace, at least for now.   It's not easy for a film to make it as a high-temperature capacitor.  Most applications are in high-frequency inverters so the dissipation factor must be low as well as having a stable dielectric constant and good breakdown voltage. 

http://www.benicewiczgroup.com/UserFiles/benigroup/Documents/BB90.pdf   Proof that research into better dielectrics goes on. 

Polybenzimidazole (PBI), Celazol:  PBI has been called one of the toughest polymers there is.  It has extreme heat resistance, to 300C or more, and it has been much used in things like aerospace hardware, fuel cell membranes and fire protection gear.  It is replacing PPK and Ryton is semiconductor processing hardware.  NASA tested this material in capacitors in the 90s but nothing seems to have come of it. 

Polyaryletherketone (PEEK):  Widely used in industry for its heat and chemical resistance.  Some lab work done on the dielectric properties but little else known.  At least one manufacturer mentions it as a possible capacitor material. 

Teflon fluorinated ethylene propylene (FEP):   These caps, film-foil, are out there (Solen) but they are expensive and the market seems to be very small.  Low dissipation constant like PTFE. 

Teflon PFA, perfluoroalkoxy:  Hyflon, some lab work done, but little else known.  Mianyang Prochema Commercial Co.,Ltd. says they make the capacitor grade film but I have never seen any capacitors. 

Teflon ETFE, ethylene-tetrafluoroethylene:  Some lab work done, but little else known. 

Teflon AF:  Some lab work done, but little else known.

ECCtreme™ ECA 3000 fluoropolymer:  A new DuPont perfluoropolymer.  Good mechanicals, high temperature (to 300C) and very low dissipation factor.  Used as wire insulation for now. 

Less known fluorinated plastics, such as modified PTFE (with some side chains), polyvinyl flouride PVF, and others:  Generally better mechanical properties than PTFE.  Some should have the electrical and temperature properties to make them attractive as capacitor films.  Economics probably works against this however as it almost always does. 

Hydroxylated polystyrene:  Some lab work done, little else known.   

Polyparaphenylene benzobisthiazole:  Little known.

Polyamide, Nylon:  Some lab work done, little else known. 

Exotic ceramic materials containing elements like scandium and lanthanum:  Ceramic capacitor makers are still pursuing materials for better performance at higher temperatures.
 
Polybenzoxazole (PBO) and its relatives:  Some lab work, little else known.  Stronger than Kevlar and very high temperature resistance. It was used in bullet-resistant vests for a time but it's subject to environmental degradation. 

Rexolite:  Crosslinked polystyrene, this material has been around since WW II.  Interesting dielectric characteristics with wide usage in microwave applications through as high as 500 GHz.  Temperature range only to about 100C which would prevent its use in capacitors. 

Aromatic polyureas:  Said to have properties that might make these polymers useful for energy storage, especially at high temperature.  Manufacturing procedure might prevent economic high volume production however. 

Nomex:  Some lab work done, little else known. 

Fluorene polyester (fluorine isophthalate terephalate):  Dearborn (with the Air Force and others) was looking at this stuff.  Dearborn now has 250C film capacitors of this dielectric, but I don't know about actual availability.  The cost of the base film is said to be very high. 

Glass:  Glass capacitors are already in use, but Penn State has been doing research with 10 um glass film for high-power capacitors. 

Aluminum oxynitride:  More at home in bullet-proof glass.  Some availability from a company called K Systems.  High temperature. 

Diamond-like carbon:  This material has been researched for some years.  Some possible availability from K Systems.  High temperature. 

Boron nitride in several forms:   Some lab work done, little else known.

Beryllium oxide:  Some lab work done, little else known. 

Titanium dioxide:  Titanium dioxide has a higher dielectric constant than aluminum oxide so it's seen as a possible electrolytic capacitor of the future, maybe on aluminum foil.  I haven't seen any actual parts but there is fair amount of research and a number of patents (some going back over 50 years).  Case Western Reserve University is active in this research. 

Perovskite materials:  High dielectric constant, some work done over the last two decades, no idea what the state of development is now.   

Sapphire:  This is one of the few early exotic dielectrics to make the cut thanks to its GHz properties.  Now used in trimmer capacitors. 

Polymethylpentene (PMP):  Some work done, little else known. 

Fluorophlogopite mica:  A fluorinated form of phlogopite.  Higher temperature than muscovite but with muscovite's electrical properties.  No one is making capacitors with it yet but it seems to have some promise. 

Synthetic micas:  A variety of synthetic micas have have been investigated but little known. 

Modified films:  There is lot of research into modifications of existing polymers such as BOPP and PVDF, as well as closely related polymers.  For example, by combing them with other polymers or nanofillers, it is hoped get a dielectric that has a better combination of properties than any one polymer

http://www.aerovox.com/Products/DefibrillatorCapacitors.aspx  Dielectric not stated. 
http://www.ga.com/series-dq-defibrillator-capacitors