Your Web News in One Place

Help Webnuz

Referal links:

Sign up for GreenGeeks web hosting
June 26, 2021 06:09 pm GMT

Dealing with huge xml-like files containing illegal characters

Sometimes you have to deal with files that look pretty like xml. But htey are not, because contain a lot of illegal characters. Such files usually made by just concatenating strings and not verifying angainst any schema. When these files are comparably small, it's not a big deal to use regex to replace all those characters, but for really big files it's not very convinient.

The idea was to read file element by element and if next part is not a valid xml, then I could clean all illegal characters and use well-formed xml for my needs.

public IEnumerable<XElement> GetElement(string filePath, string elementName){    using var reader = XmlReader.Create(filePath, new XmlReaderSettings { CheckCharacters = false });    reader.MoveToContent();    while (reader.Read())    {        if (reader.NodeType == XmlNodeType.Element && reader.Name == elementName)        {            var str = reader.ReadOuterXml();            XNode node;            try            {                node = XElement.Parse(str);            }            catch (XmlException)            {                var pattern = @"&#x((10?|[2-F])FFF[EF]|FDD[0-9A-F]|[19][0-9A-F]|7F|8[0-46-9A-F]|0?[1-8BCEF]);";                var regex = new Regex(pattern, RegexOptions.IgnoreCase);                var fixedStr = regex.Replace(str, string.Empty);                node = XElement.Parse(fixedStr);            }            if (node is XElement el)            {                yield return el;            }        }    }}

Important note:
To skip checking for invalid characters, you should pass to XmlReader settings XmlReaderSettings { CheckCharacters = false } so it can omit checks and give me possibility to cleanup input string.


Original Link: https://dev.to/t3mplar/dealing-with-huge-xml-like-files-containing-illegal-characters-1bh7

Share this article:    Share on Facebook
View Full Article

Dev To

An online community for sharing and discovering great ideas, having debates, and making friends

More About this Source Visit Dev To