Support selection of saves from sub-directories

Discussion of all aspects of the game engine, including development of new and existing features.

Moderator: Forum Moderators

Post Reply
demario
Posts: 130
Joined: July 3rd, 2019, 1:05 pm

Support selection of saves from sub-directories

Post by demario »

After having played with saves quite a bit, I am still surprised by the lack of support for selecting saves from sub-directories.

There are two file explorers in wesnoth: the load save dialog (SP, MP) and the map editor file dialog (load, save). They do quite different things.
screenshot of SP/MP load save dialog:
The load save dialog (src/gui/dialogs/game_load.cpp, data/gui/window/game_load.cfg) is used to search, load and delete saves. It doesn't show the sub-directories (but show all types of files, with non-save files showing invalid/corrupted when selected). The search function is based on a text box to input the criteria, with different words can be separated with spaces and input at any order. After reading the code, I realize it also includes a button ("Open saves folder") at bottom-left corner, which starts a file explorer (I need to push the button 2,3 times to open it ;) ). The button is small and without label and I barely ever noticed it. But it is not really useful as file or directory selected in the explorer do not transfer back to wesnoth for selection. It might be a bug or the feature is just to allow easy copying saves from place of storage to the only directory that wesnoth reads saves from.
screenshot of map editor file dialog:
The map editor file dialog (src/gui/dialogs/file_dialog.cpp, data/gui/window/file_dialog.cfg) is used to load and delete map files (no search) but also to save maps. It is fair to say that the dialog is designed mainly around saving files: there is no search on existing files (need to scroll down), the text box is used to input new file name (not used as a filter to search for existing files), scrolling down the list of files fills the input text box with selected file name (to easily support adding suffix to existing file names).
There is an additional feature to add bookmark (shortcut) to outside directories in the map editor file dialog for direct access. I believe this is to meet the specific needs of map and campaign designers to add their add-ons as bookmarks.

So my idea is to take useful features from the map editor file dialog and add them to the SP/MP load save dialog:
game_load_subdirs1.png
[Data shown on the screenshot are coming from Maboul's replay package]
another screenshot of changed dialog:
  • It supports:
    - search based on names of save files (multiple keyword)
    - preview of save summary
    - loading, deleting save files
    - browsing through sub-directories to perform load/delete actions
  • It excludes:
    - bookmark management
    - save action
  • Known bugs:
    - Open saves folder (this is a regression bug)
    - No column sort (disable until it works well)
    - Don't show sub-directories if main directory contains no file
    - Inconsistencies between the search/sort and the presence of directories
In attachment is the file containing the changes to apply to BFW1.14 to add support to sub-directories in load game dialog.
[edit] Patch is back up :mrgreen: [/edit]
The changes are compatible with the changes to support replaying old saves (1.4-1.12 1v1).
Attachments
Support_selection_of_saves_from_sub_directories.v2.diff
Apply over clean BFW1.14 (or along support for.12-1.4 replays), patch -p2, build with cmake
(48.95 KiB) Downloaded 97 times
Last edited by demario on August 28th, 2021, 10:35 am, edited 3 times in total.
"simply put, it's an old game"T — Cackfiend
User avatar
Atreides
Posts: 1039
Joined: March 30th, 2019, 10:38 pm
Location: On the 2nd story of the centre village of Merwuerdigliebe turning the lights on and off

Re: Support selection of saves from sub-directories

Post by Atreides »

Funny, I just found out that you can't load replays from a subdir a few days back. I'd moved dl replays and when I went to load some I was surprised that it wasn't possible. I've posted in the past that I wished that saves and replays were stored seperately by default.
demario
Posts: 130
Joined: July 3rd, 2019, 1:05 pm

Re: Support selection of saves from sub-directories

Post by demario »

This is an update of the dialog to select saves to load. This version is fixing some issue in the original search on file names.
original search results for "ladd 2015":
The search was done on the file names as displayed in the dialog. So files whose name were shortened for display (to fit into screen size) would not match any pattern present in the end of the name. For example in the screenshot, if the ladd search word was fully input into ladder, the second match would disappear from the result as the remaining of the word was replaced by ellipsis .... There are already several matches not shown because the ellipsis has cut the name before the second 'd' of ladder. To fix this problem, it is required to apply the search on the underlying data rather than the data shown in the list.

The new version is fixing this problem.
new search result for "ladder 2015":
You can see other changes in the display too:
  • all files whose full name are matching the search words are shown
  • given that the version is old (1.10) the option to show replay is enabled by default
  • both sides are listed in the summary as this is a save to replay
Now that the search is done on the original data rather than the data in the widget, we could extend the search on other items than the file name: player names, leader types, map name, ... One use case is certainly the faction name ("all ladder games from 2015 on freelands, where knalgans face Northerners as player 1") but this information is not in the summary yet. So additional preparation work would be needed.
I think about a syntax like "2015 ladder s: freelands p1: Knalgans p2: Northerners" for search criteria. Everything before s: applied to file name, everything after applies to summary data, with words after p1/p2: applying to the corresponding side.
Atreides wrote: I've posted in the past that I wished that saves and replays were stored separately by default.
I think that is a fair request, the saves have a lifespan of a couple of days unless you're making a break while the replays could be kept for years.
Attachments
Support_selection_of_saves_from_sub_directories.v3.diff
Apply over clean BFW1.14 (or along support for.12-1.4 replays), patch -p2, build with cmake
(61.66 KiB) Downloaded 107 times
"simply put, it's an old game"T — Cackfiend
User avatar
octalot
General Code Maintainer
Posts: 777
Joined: July 17th, 2010, 7:40 pm
Location: Austria

Re: Support selection of saves from sub-directories

Post by octalot »

1.16 added a directory-selector to the file dialog, for loading files from old versions (it's hidden unless it finds other directories to show). That framework can also be used for subdirectories; while it's not as flexible as your version, it's also a much smaller change for 1.16.

Prototypes for 1.16 and 1.14 (the 1.14 branch includes a backport of the 1.16 changes):
https://github.com/stevecotton/wesnoth/ ... bdirs_1_16
https://github.com/stevecotton/wesnoth/ ... bdirs_1_14

In the existing load dialog, it seems searching already uses the full name instead of the truncated one.
demario
Posts: 130
Joined: July 3rd, 2019, 1:05 pm

Re: Support selection of saves from sub-directories

Post by demario »

Thank you for the feedback. It is good to hear that people are experimenting with different solutions.
octalot wrote: September 5th, 2021, 6:06 pm Prototypes for 1.14 (the 1.14 branch includes a backport of the 1.16 changes):
https://github.com/stevecotton/wesnoth/ ... bdirs_1_14
I am illiterate in github. Is there a way for me to retrieve a diff file for the changes made on this branch
I am just guessing there are three commits (8f80af6, 4abe125, 659a5e4) based on revision b879239.
User avatar
octalot
General Code Maintainer
Posts: 777
Joined: July 17th, 2010, 7:40 pm
Location: Austria

Re: Support selection of saves from sub-directories

Post by octalot »

Are you already using a Git clone of the main Wesnoth repo? If you are then you could either add my fork as a remote, alternatively git fetch https://github.com/stevecotton/wesnoth show_save_subdirs_1_14 will download just that branch and make a local branch called FETCH_HEAD.

If you're working from the release tarballs instead then (apart for recommending you clone the repo), the patches are attached.
Attachments
0003-Fixup-show-subdirs-for-backporting-to-1.14.patch
(1.59 KiB) Downloaded 100 times
0002-WIP-Show-subdirectories-in-the-load-game-dialog-reus.patch
(4.14 KiB) Downloaded 93 times
0001-WIP-Add-other-versions-of-Wesnoth-to-the-Load-Game-d.patch
(42.21 KiB) Downloaded 105 times
User avatar
egallager
Posts: 568
Joined: November 19th, 2020, 7:27 pm
Location: Concord, New Hampshire
Contact:

Re: Support selection of saves from sub-directories

Post by egallager »

demario wrote: September 6th, 2021, 12:35 pm Thank you for the feedback. It is good to hear that people are experimenting with different solutions.
octalot wrote: September 5th, 2021, 6:06 pm Prototypes for 1.14 (the 1.14 branch includes a backport of the 1.16 changes):
https://github.com/stevecotton/wesnoth/ ... bdirs_1_14
I am illiterate in github. Is there a way for me to retrieve a diff file for the changes made on this branch
I am just guessing there are three commits (8f80af6, 4abe125, 659a5e4) based on revision b879239.
One neat feature GitHub has is that you can just attach .diff to the end of any commit url to get the diff version of it, so for those 3, it'd be:
https://github.com/stevecotton/wesnoth/commit/8f80af61d71d2afa3e9bb17642b554d3a943c4b6.diff
https://github.com/stevecotton/wesnoth/commit/4abe1259b8599f920d68ab9a1e8df1288b5ffb16.diff
https://github.com/stevecotton/wesnoth/commit/659a5e4869ececaca3a19cdd995b210da2a7325b.diff
You can also use the "compare" tool to generate diffs between arbitrary commits:
https://github.com/wesnoth/wesnoth/compare
So, to use that to put the 3 of them together, you get:
https://github.com/wesnoth/wesnoth/comp ... 0da2a7325b
...and the .diff suffix can be added to this URL, too:
https://github.com/wesnoth/wesnoth/comp ... 7325b.diff
demario
Posts: 130
Joined: July 3rd, 2019, 1:05 pm

Re: Support selection of saves from sub-directories

Post by demario »

egallager wrote: September 6th, 2021, 7:11 pm One neat feature GitHub has is that you can just attach .diff to the end of any commit url to get the diff version of it. [...]
You can also use the "compare" tool to generate diffs between arbitrary commits: https://github.com/wesnoth/wesnoth/compare
So, to use that to put the 3 of them together, you get:
https://github.com/wesnoth/wesnoth/comp ... 7325b.diff
Cheers egallager, you're the man!


Based on the diff file, I can compile and it runs smoothly for me. Here are some screenshots of the changes for reference.
show_save_subdirs_1_14.main.png
The sub-directories do not show in the file list (nor go-to-parent ..). A drop-down list box next to the search input text with Current Version.
Inside the drop down list box:
Getting inside the box, the sub-directories are shown together with the different save directories from past versions that can be found.
When selecting a past version:
Shows the save files in the save directory from the selected version. The delete button is disabled when browsing save directory from a past version.
octalot wrote: September 5th, 2021, 6:06 pm That framework can also be used for subdirectories; while it's not as flexible as your version, it's also a much smaller change
All in all, the change looks good and works perfect. The drop-down list box is helping saving space in this dialog that is already pretty full. it would help for the tiny-gui (mobile phone UI) and is a solid alternative of the bookmark management as implemented in the map file dialog. The save directory from past versions could be interpreted as default bookmarks created automatically when present.

Still I like to see the sub-directories in the list of files as they are part of the "Current Version" for me and it is easy to walk them through with double click and go back (rather than hitting back the drop-down list). I also think that the sub-directories being in the list, they are also the target of the search which may be good in some cases. For example, if a sub-directory showed only if it contains at least a file that matches the search (not implemented yet).

At the end of the day, are we more likely to have past replays in the original save directory (meaning past versions used to be running on that computer or that the user's home directory was recovered from a past install) or to be stored in a sub-directory of the current version after download? I am not clear on people habits. Alternative settings (cloud storage, apple cloud, Steam storage, ...) may also impacts the benefits.

On the code I see a couple of changes in src/save_index.cpp that I was lead to modify too. I trust your changes more than mine and I will see if I can retrofit your code in my version. My changes were a proof-of-concept and I did a bad job at showing that most of the code in the first diff is not new code but copy/paste from src/gui/dialog/file_dialog.cpp (you can do a diff between the two files to show common code). If we wanted to use that design, I think we can encapsulate the code in functions common to both map file and save file dialogs, which would look better.
octalot wrote: September 5th, 2021, 6:06 pm In the existing load dialog, it seems searching already uses the full name instead of the truncated one.
When applying the diff file from github, the search is still applied on the displayed string (truncated file names), as you can guess from the third screenshot.
Are you talking about another "existing" dialog?
octalot wrote: September 6th, 2021, 12:55 pm If you're working from the release tarballs instead
Yes, my working copy is out of the release tar file downloaded from sourceforge.
User avatar
octalot
General Code Maintainer
Posts: 777
Joined: July 17th, 2010, 7:40 pm
Location: Austria

Re: Support selection of saves from sub-directories

Post by octalot »

demario wrote: September 7th, 2021, 7:32 am
octalot wrote: September 5th, 2021, 6:06 pm In the existing load dialog, it seems searching already uses the full name instead of the truncated one.
When applying the diff file from github, the search is still applied on the displayed string (truncated file names), as you can guess from the third screenshot.
Are you talking about another "existing" dialog?
It seems I only tested that on 1.16, sorry for the confusion. It was fixed for 1.16 in fd51cdcfeb8ca6b4b7fc74b2c392fdbf1b985e46.
demario
Posts: 130
Joined: July 3rd, 2019, 1:05 pm

Re: Support selection of saves from sub-directories

Post by demario »

So I did a mashup of the 2 solutions and it looks like this:
mashup.main.png
The code is in attachment and can be applied to a clean state BFW1.14 code (I used BFW 1.14.14, patch -p2 and cmake for compilation but other should work too). It is based on octalot's show_save_subdirs_1_14 (and so conflicting with SPDE) where I made additional change to show sub-directories in file list with the ability to navigate the directory tree. I have limited the tree to the save directory of the selected version (no .. present in main screen). The search is working but the sort is disable.
I have kept the cleanup in code structure as it was helpful to me to understand the code and implement the changes (even if it makes it a bigger change). This includes moving the data manipulation code into _model class (following MVC-like approach).

There are still some bug (sort, delete button, long directory names, ...) but I will leave it sit there for a while.

The features are:
show past version only when present:
select past version from drop down list:
navigate through subdir in the file list:
I see some possible improvements:
  • do not allow delete of directories
  • allow delete into sub-directories
  • code internals: completed removal of current_dir_ and games_ from game_load class
  • code internals: possibly move the save_index_manager_ to filelist_data_model class
Attachments
Support_selection_of_saves_from_sub_directories.v4.diff
Apply over clean BFW1.14 (or along support for.12-1.4 replays), patch -p2, build with cmake
(65.71 KiB) Downloaded 89 times
"simply put, it's an old game"T — Cackfiend
demario
Posts: 130
Joined: July 3rd, 2019, 1:05 pm

Re: Support selection of saves from sub-directories

Post by demario »

/me wrote:September 5th, 2021, 8:11 am One use case is certainly the faction name ("all [..] games [..] on [Swamp of Dread], where Knalgans face Northerners as player [2]") but this information is not in the summary yet. So additional preparation work would be needed.
I think about a syntax like ["s: swamp p1: Northerners p2: Knalgans"] for search criteria. Everything before s: applied to file name, everything after applies to summary data, with words after p1/p2: applying to the corresponding side.
[The data used here is server saves still from Maboul's replay packages]
I have reused the option parsing boost library for this use case. So the syntax is rather "--s Swamp --p1 Northerners --p2 Knalgans".

So that it ends up looking like this:
summary_search.png
Each search "option" take a comma-separated list of keywords.
So it is possible to narrow the search:
The initialization process (it seems you need to remove the save_index) is very long. It took me 20min to load the summary for the ~2000 saves in the archive. The summary is a subset of the save WML used to display the information about the save shown on the left panel of the screen.
Post Reply