On behalf of Charles Riley:

---------- Forwarded message ----------
From: Riley, Charles <[log in to unmask]>
Date: 23 February 2016 at 05:37
Subject: [camms-ccaam] Common encoding errors
To: "[log in to unmask]" <[log in to unmask]>, "
[log in to unmask]" <[log in to unmask]>, "[log in to unmask]" <
[log in to unmask]>, "[log in to unmask]" <
[log in to unmask]>, "[log in to unmask]" <
[log in to unmask]>, "[log in to unmask]" <
[log in to unmask]>


Hi all,



This is something I’ve noticed happening with somewhat regular, and
probably increasing occurrence lately:  a class of problems with records
containing either escaped entity references from HTML or XML (like
‘&nbsp;’), or accented characters that have become corrupted in a data
migration (like ‘français
<https://openlibrary.org/works/OL10004281W/Les_archets_français>‘).  I was
asked by another librarian if I could point them to any resources that deal
with this class of issues, and rounded up a few that I thought would be
good to share.  Here’s what I came across, in terms of examples and
explanations for some of the more common cases:



http://markmcb.com/2011/11/07/replacing-ae%E2%80%9C-ae%E2%84%A2-aeoe-etc-with-utf-8-characters-in-ruby-on-rails/



https://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references
(But treat this list with caution in using it to search; there will be
false positives for a search for ‘amp;’, for example.)



http://www.i18nqa.com/debug/utf8-debug.html (See also associated links on
this page.)



Hope this helps!



Charles Riley



*Charles Riley*

*Interim Librarian for African Studies and Catalog Librarian*

*Sterling Memorial Library*

*Yale University*



*[log in to unmask] <[log in to unmask]>*

*(203)432-7566 <%28203%29432-7566> or (203)432-9301 <%28203%29432-9301>*







-- 
Andrew Cunningham
[log in to unmask]