file names, directories and case sensitivity

[norbert]

I don't know if Windows uses case sensitivity for file names and directories, but it's annoying that quite frequently users upload packages to the add-on server that simply don't work and require me to rename file names and/or directories because their names don't match those used in the code. This really happens quite frequently, I'm no longer surprised to see "Error" messages when I select scenarios or campaigns. Can the devs make sure those users also get an error message?

[Edit: A similar problem is that Windows users frequently type the path separator characters for, for example, web icons with backslashes instead of slashes (which gives them an icons/missing-image.png).]

[zookeeper]

Well, we delete broken add-ons from the server from time to time anyway, so if you find a broken one you can just PM the author and/or ask on #wesnoth-dev for someone to delete it.

[norbert]

Well, we delete broken add-ons from the server from time to time anyway

Okay, but this means work for humans and possibly throwing away good content. If the application would use case sensitivity on all platforms, this would help prevent broken add-ons being uploaded and give the users more (possibly good) content. An add-on developer might even give up on his/her work when nobody takes the time to inform the dev (why) the add-ons isn't working and it's not being used - or because people complain about errors that cannot be replicated on certain platforms.

[silene]

If the application would use case sensitivity on all platforms,

Please enlighten us. I for one have no clue on how to force Windows to forget about case insensivity. Even Cygwin can't do it, unless you enable case sensitivity system-wide. (There is a registry setting for that.) Breaking every other Windows applications just for the sake of Wesnoth seems a bit overkill.

[norbert]

Please enlighten us.

I can't, if the problem is Windows...

I don't know if Windows uses case sensitivity for file names and directories, but it's annoying that [...]

I guess no solution for this problem for now.

[Sapient]

Or you could "fix" the Linux version to be case insensitive.

[tsr]

Hm, couldn't you force it from wesnoths side anyway?

When reading WML-paths require that path and file names are in lowercase and using forward slashes as separators?

Raise an error if either (or both) of the following occur:
- the wml-preprocessor encounters a path like "~/add-ons/Blaha/UtIls.cfg"
- the wml-preprocessor checks the actual path and file names for uppercase characters and finds one

Whit an instructive error-message in the form of something like:
For compatability reasons (across several plattforms) Wesnoth has decided that:
- All files and directories must only use lowercase letters, numbers and underscores in their names.
- All references to files must only use lowercase letters, numbers and underscores in their values.
- All references to paths must use the forward slash '/' as the directory-separator

Sure, it will make the preprocessor slower, but it might be worth it...

Of course I have no idea how wesnoth reads files and/or file-references and if it is even possible to get the real (I mean considering case) name of a file in windows.

Another option would be to translate all names and references when uploading to the add-ons server, but that might be confusing for UMC-developers and would still allow breakage in un-uploaded content (like forum previews).

/tsr

[Momar]

I didn't program C for a very long time (I mean "seriously") but may be it can use "findfirst" functions. [acronym="if I remember correctly]IIRC[/acronym], the "findfirst" structure stores the actual name of the file in its actual case even in DOS/Windows systems. Of course it will make file opening slower as it "finds" the file and checks the name.

Something like this:

Code: Select all

#include <dir.h>
#include <stdio.h>

FILE* MyWesnothFileOpen (char *FileName, char *FileMode)
{
  struct ffblk FileFind;

  if (findfirst (FileName, &FileFind, FA_ARCH)
     || strcmp (FileFind.ff_name, FileName) != 0)
  /* Didn't find the file or the name isn't "the same". */
    return NULL;
  return fopen (FileName, FileMode);
}

[Sapient]

Rather than slowing down all valid file lookups on Windows to allow them to artificially fail, wouldn't it make a lot more sense to handle the (presumably far less frequent) case of failed file lookups on Linux systems? To handle it, you can simply unwind the path until it is valid then load each section of the path by checking for a differently-cased version. Any add-ons with files or directories differing only by case would be invalid for cross-OS compatibility purposes, regardless.

[silene]

To handle it, you can simply unwind the path until it is valid then load each section of the path by checking for a differently-cased version.

Please define "differently-cased" version, from an implementation point of view. For instance, are "i" and "I" the same letter with difference cases? Depending on the locale, they aren't! Case insensivity is a can of worm; I would much prefer slowing Wesnoth on Windows to check for case sensitivity rather than reimplementing in Linux the whole codepage mechanism of Windows.

Note that, in most cases, I don't even think there would be a slowdown on Windows. Indeed, Wesnoth is already visiting the directories (e.g. binary_path) in order to find the corresponding file. I'm sure someone could come with an implementation that doesn't incur more system calls. So Windows developers, how do you change the following Wesnoth function so that it tests for both existence and case? (Usage of Windows API is allowed, obviously.)

Code: Select all

bool file_exists(const std::string& name)
{
       struct stat st;
       return ::stat(name.c_str(), &st) == 0;
}

[Iris]

[Edit: A similar problem is that Windows users frequently type the path separator characters for, for example, web icons with backslashes instead of slashes (which gives them an icons/missing-image.png).]

Disregarding the "web" part (which I have no idea what it exactly means), I'd suggest two solutions:

1) Disallowing backslashes in file system method arguments.
2) Translating backslashes in file system method arguments.

Of course (2) would mean even more string replacements at runtime and would encourage laziness on part of the content authors. It could also make bogus paths such as "C:\Users\Foobar\Documents\blah.png" less obvious — and it would also look out of place without universal case-insensitivity. So I'd go for (1).

(Also, I seem to recall that FAT32 filesystems are not case-preserving on Windows 9x, but since we don't support those anymore they'd be out of the question. After skimming the Wikipedia article on FAT it seems that FAT32 is case-preserving on the Windows NT family. I also remember that NTFS is case-preserving.)

Another solution to consider would be a upload-time check from the add-ons server that took advantage of the add-on packages' format used for transmitting over the network — WML data which could be searched for strings that look like filenames. This would probably impose a lot of extra load in the add-ons server though, since not only the add-ons' own directories and files would need to be checked for case matches, but also Wesnoth's core datadir.

A different solution would be a periodic (daily, weekly, etc.) check run on all the add-ons in the server (maybe using wmllint*) that delivered results to be viewed by everyone who wanted to take action on them.

(* I don't like hate wmllint. ESR has proposed making wmllint-awareness a requirement for keeping content on the add-ons server before, though.)

EDIT: in case you are desperate and want to be fancy, you can also create a VFAT filesystem image on your case-sensitive Linux-compatible filesystem and mount it as your .wesnoth/data/add-ons directory, by the way. I know it sounds ridiculous, but I once did that for running a program that ran on Linux but used data files that were only tested on Windows. No, it won't solve the problem for mismatched case on references to core data files. You could create another image for /usr/share/wesnoth though!

[Sapient]

To handle it, you can simply unwind the path until it is valid then load each section of the path by checking for a differently-cased version.

Please define "differently-cased" version, from an implementation point of view. For instance, are "i" and "I" the same letter with difference cases? Depending on the locale, they aren't! Case insensivity is a can of worm; I would much prefer slowing Wesnoth on Windows to check for case sensitivity rather than reimplementing in Linux the whole codepage mechanism of Windows.

By "differently cased" I mean of course that ASCII a-z can be interchangeable with A-Z. That , by the way, is one of the weakest straw man arguments I have ever heard and has no absolutely no bearing on any usermade content I am aware of.

[silene]

By "differently cased" I mean of course that ASCII a-z can be interchangeable with A-Z.

You really missed my point, didn't you? Windows on a latin5 system doesn't interchange i and I! Perhaps your Windows is ASCII only, but this is not the case for anyone out there!

[Sapient]

The code would not even be active on Windows, but on case sensitive OS when a file was not found, so the point whatever it was is meaningless. Also, any add-ons with files or directories differing only by case would be invalid for cross-OS compatibility purposes.

Remember the original problem being discussed is getting the usermade content to work for everyone, so it just needs to work for all the practical cases -- without annoying content creators or needlessly slowing down normal file operations.

[silene]

The code would not even be active on Windows, but on case sensitive OS when a file was not found, so the point whatever it was is meaningless. Also, any add-ons with files or directories differing only by case would be invalid for cross-OS compatibility purposes.

Again, you are missing my point; are you even trying? Let's suppose there is a UMC containing a file with "I" in its name but it is referred with "i" in WML (or the contrary). Then there are some Windows systems where the file won't be found. So we want to detect this kind of issue, and the only way I can imagine is to enforce case sensitivity.