Non-coding transcripts originating from bidirectional promoters have been reported in a wide range of organisms. In yeast, these divergent transcripts can be subdivided into two classes. Some are designated Cryptic Unstable Transcripts (CUTs) because they are terminated by the Nrd1-Nab3-Sen1 pathway and then rapidly degraded by the nuclear exosome. This is the same processing pathway used by yeast snoRNAs. Whereas CUTs are only easily observed in cells lacking the Rrp6 or Rrp47 subunits of the nuclear exosome, Stable Uncharacterized Transcripts (SUTs) are present even in wild-type cells. Here we show that SUTs are partially susceptible to the nuclear exosome, but are primarily degraded by cytoplasmic 5' to 3' degradation and nonsense-mediated decay (NMD). Therefore, SUTs may be processed similarly to mRNAs. Surprisingly, both CUTs and SUTs were found to produce 3' extended species that were also subject to cytoplasmic degradation. The functions, if any, of these extended CUTs and SUTs are unknown, but their discovery suggests that yeasts generate transcripts reminiscent of long non-coding RNAs found in higher eukaryotes.