Unicode

BlitzMax Forums/BlitzMax Programming/Unicode

Dreamora is correct in saying that the Blitzmax file system doesn't work with certain foreign (as in unicode) filenames

Yep - at present BlitzMax can store unicode strings internally but can't access files with unicode file names.

Seb steps in and says "here we go - it's working!"?

The problem lies with the standard C functions imported in Pub.Stdc - Microsoft Windows have different commands for handling unicode (wide character) strings and so these have to be used instead.

For example,

ANSI:
fopen( file$z, mode$z )
Windows Unicode Equivalent:
_wfopen( file$w, mode$w )
However, Mac OS X and Linux (I think) took a different tack - they allow Unicode file paths to be passed into the same old C functions that used to only handle ANSI characters, and they should be correctly distinguished between and handled by the OS. All that is needed in this case is to change the Extern declarations so that they pass wide-character (unicode) string ($w) instead of standard ANSI C strings ($z), e.g.

fopen( file$w, mode$w )
However we need to ensure that the older legacy Windows operating systems play nice with Unicode - Windows 95/98/Me were built for use with the older, more limited, ANSI character set and so Unicode support on these platforms is complicated - in addition, FAT and FAT32 (the most common file systems used with these versions of Windows) don't support Unicode filenames anyway!?!?

Can the file system (and thus bmax as a whole) handle unicode character sets or not?

Most modern file-systems can (for example, NTFS and OS X EFS, but not older file-systems such as FAT or FAT32). However, the BlitzMax modules need to be tweaked if unicode is to be supported by apps (regardless of whether Unicode is supported by the file-system or not).

Is it a hack or something more stable that might make it (soon) to a new BlitzMax release?

I've been experimenting with different approaches - the tweaks seems rock-solid on Windows 2000+, but Windows 9X is choking on it at the moment - it doesn't even like the new Unicode MaxGUI.Win32MaxGUIEx module for the same reason.

In the end it might end up being a toss-up between supporting Unicode, or supporting Windows 9X... Only Mr Sibly can make that decision, still, let's see how it goes though, hopefully I have a few more tricks up my sleeve yet.