Do you do sometimes deal with MARC in the MARC8 character encoding?  Do 
you deal with software that converts from MARC8 to UTF8?

Maybe sometimes you've seen weird escape sequences that look like HTML 
or XML "character references", like, say "‏".

You, like me, might wonder what the heck that is about -- is it 
cataloger error, a catalgoer manually entered this or something in 
error? Is it a software error, some software accidentally stuck this in, 
at some part in the pipeline?

You can't, after all, just put HTML/XML character references wherever 
you want -- there's no reason "‏" would mean anything other than 
&, #, x, 2, etc, when embedded in MARC ISO 2709 binary, right?

Wrong, it turns out!

There is actually a standard that says you _can_ embed XML/HTML-style 
character references in MARC8, for glyphs that can't otherwise be 
represented in MARC8. "Lossless conversion [from unicode] to MARC-8 
encoding."

http://www.loc.gov/marc/specifications/speccharconversion.html#lossless

Phew, who knew?!

Software that converts from MARC8 to UTF-8 may or may not properly 
un-escape these character references though. For instance, the Marc4K 
"AnselToUnicode" class which converts from Marc8 to UTF8 (or other 
unicode serializations) won't touch these "lossless conversions" (ie, 
HTML/XML character references), they'll leave them alone in the output, 
as is.

yaz-marcdump also will NOT un-escape these entities when converting from 
Marc8 to UTF8.

So, then, the system you then import your UTF8 records into will now 
just display the literal HTML/XML-style character reference, it won't 
know to un-escape them either, since those literals in UTF8 really _do_ 
just mean & followed by a # followed by an x, etc. It only means 
something special as a literal in HTML, or in XML -- or it turns out in 
MARC8, as a 'lossless character conversion'.

So, for instance, in my own Traject software that uses Marc4J to convert 
from Marc8 to UTF8 -- I'm going to have to go add another pass, that 
converts HTML/XML-character entities to actual UTF8 serializations.  Phew.

So be warned, you may need to add this to your software too.