Extend the characters allowed for selectors

[Mirchev98]

Hello! Using the latest version of SmartFormat, I encountered an issue, where my template cannot be processed when it contains Cyrillic letters.

The code:

using SmartFormat;
namespace TestSmartFormat
{
	public class AttributeData
	{
		public string DisplayText { get; set; }
	}

	public class Document
	{
		public Dictionary<string, AttributeData> ArchiveAttributes { get; set; }
	}

	public class UnifiedDocumentData
	{
		public Document Document { get; set; }
	}

	class Program
	{
		static void Main(string[] args)
		{
			// Initialize sample data with Cyrillic keys
			var data = new UnifiedDocumentData
			{
				Document = new Document
				{
					ArchiveAttributes = new Dictionary<string, AttributeData>
					{
						{ "София", new AttributeData { DisplayText = "София" } },
						{ "Пловдив", new AttributeData { DisplayText = "Пловдив" } }
					}
				}
			};

			// Define templates
			string template1 = "{ArchiveAttributes[София].DisplayText}";
			string template2 = "{ArchiveAttributes[Пловдив].DisplayText}";

			var formatter = Smart.CreateDefaultSmartFormat();

			try
			{
				// Perform formatting
				string result1 = formatter.Format(template1, data.Document);
				string result2 = formatter.Format(template2, data.Document);

				// Display results
				Console.WriteLine($"Template 1 Result: {result1}"); // Expected: София
				Console.WriteLine($"Template 2 Result: {result2}"); // Expected: Пловдив
			}
			catch (Exception ex)
			{
				Console.WriteLine($"Unexpected Error: {ex.Message}");
			}
		}
	}
}

The error:
Unexpected Error: The format string has 5 issues:
'0x421': Invalid character in the selector, '0x43E': Invalid character in the selector, '0x444': Invalid character in the selector, '0x438': Invalid character in the selector, '0x44F': Invalid character in the selector
In: "{ArchiveAttributes[София].DisplayText}"
At: -------------------^^^^^

However, when I change the following part to contain only English, it works perfectly well:

var data = new UnifiedDocumentData
			{
				Document = new Document
				{
					ArchiveAttributes = new Dictionary<string, AttributeData>
					{
						{ "Sofia", new AttributeData { DisplayText = "София" } },
						{ "Plovdiv", new AttributeData { DisplayText = "Пловдив" } }
					}
				}
			};

			// Define templates
			string template1 = "{ArchiveAttributes[Sofia].DisplayText}";
			string template2 = "{ArchiveAttributes[Plovdiv].DisplayText}";

I am unsure if I am missing something, or if it is genuinely a bug.

Thanks in advance for your help!

[axunonb]

The default character that are allowed for selectors are "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_-". This is by design.
You may, however, add more characters, if needed:

// Create a list of all Cyrillic characters
var cyrillicChars = new List<char>();
for (var c = '\u0400'; c <= '\u04FF'; c++)
{
    cyrillicChars.Add(c);
}
for (var c = '\u0500'; c <= '\u052F'; c++)
{
    cyrillicChars.Add(c);
}

var settings = new SmartSettings();
settings.Parser.AddCustomSelectorChars(cyrillicChars);
var formatter = Smart.CreateDefaultSmartFormat(settings);

With this extension, your sample will work.

Might be an enhancement to remove the valid character set, and just exclude specific reserved characters.

[axunonb]

@karljj1 What do you think about extending the valid characters for selectors in general?

[karljj1]

@karljj1 What do you think about extending the valid characters for selectors in general?

That's a good idea, especially when dealing with localization. Id expect any character to work for a selector except those that have other functions.

[Mirchev98]

@axunonb That works perfectly well, it solved the issue. Thank you!

[vdachev-david]

@karljj1 , @axunonb , how about allowing to use literals (e.g. {ArchiveAttributes["Sofia"]} in order to access the dictionary item with key "Sofia")? This way one can access dictionary items with different key types without extending the valid characters.

[axunonb]

@vdachev-david Using a numeric index is already implemented, but you mean a string index here{ArchiveAttributes["Sofia"]}?