GSoC: Sprite Sheets

[doofus-01]

Hi,

Would instances where two or more .pngs are currently used simultaneously, such as halo effects, be affected? It would be unfortunate for UMC authors/artists if the halos were merged with the mainline sprites in a spritesheet.

(My apologies if that was already addressed. I didn't see it, but then I did not really understand what tsr was asking.)

[Boucman]

more generally, frogotto's approch to describing animations is probably way too limited for wesnoth. It reduces animations to a sequence of images, but animations in wesnoth are much more than that...

sprite sliding is part of the animation engine, poison recoloring is poart of the animation engine, fade in/out is part of the animation engine, unit selection is part of the animation engine...

and by "part of the animation engine" I mean that it's entirely possible to recode a special selection animation for a given unit instead of the default...

so I wouldn't advise going the frogatto way for animation description.

[gabba]

Hi,

Would instances where two or more .pngs are currently used simultaneously, such as halo effects, be affected? It would be unfortunate for UMC authors/artists if the halos were merged with the mainline sprites in a spritesheet.

(My apologies if that was already addressed. I didn't see it, but then I did not really understand what tsr was asking.)

Can you elaborate on how halos work, or why do you think there'd be a problem?
After an image is loaded from a sprite sheet in the game engine, it acts just like an individual .png, so I don't expect any problems there.

more generally, frogotto's approch to describing animations is probably way too limited for wesnoth. It reduces animations to a sequence of images, but animations in wesnoth are much more than that...

sprite sliding is part of the animation engine, poison recoloring is poart of the animation engine, fade in/out is part of the animation engine, unit selection is part of the animation engine...

and by "part of the animation engine" I mean that it's entirely possible to recode a special selection animation for a given unit instead of the default...

so I wouldn't advise going the frogatto way for animation description.

Yeah, from the snippets of animations I saw, I had the feeling that Wesnoth's system is much more advanced, even though it may harder to use. My [frame_group] idea still stands, though.

[tsr]

Hi, I think I didn't realize where in the process this was. Just trying to throw in one more aspect of it

Ok, let's see.

@tsr: I'm not sure of what you mean exactly when you say "mix and match" the individual pngs?
Anyways, without getting into the details of your proposal, of course, these sheets could be generated at cache time. I suppose it would increase initial loading time, but as a one-time or unfrequent thing it's acceptable. I also had a slightly different take on the WML appearance (actually I want to avoid introducing anything new there), but that's not important.
The problem is that there doesn't seem to be a consensus of what people actually want from sprite sheets. If reducing disk usage is a priority, your idea wouldn't cut it...

What I meant with "not mix and match" was that you intend to line up the frames like in jetrels example and not trying to squeeze in as many non-transparent-pixels into a an as little total area as possible (like was done in really old games to preserve diskspace).

/tsr - lurks away

[doofus-01]

This is an example of a halo (from Oracle.cfg).

Code: Select all

        [frame]
            duration=50
            image="units/saurians/oracle-magic-1.png"
            halo=halo/saurian-magic-halo-1.png
        [/frame]

My concern is that the halo images can be mixed and matched with other sprites, but not if both "halo" image & "image" image are part of the sprite on a single sprite-sheet image. Is that the case?

[gabba]

Hi, I think I didn't realize where in the process this was. Just trying to throw in one more aspect of it

@tsr: I'm not sure of what you mean exactly when you say "mix and match" the individual pngs?
Anyways, without getting into the details of your proposal, of course, these sheets could be generated at cache time. I suppose it would increase initial loading time, but as a one-time or unfrequent thing it's acceptable. I also had a slightly different take on the WML appearance (actually I want to avoid introducing anything new there), but that's not important.
The problem is that there doesn't seem to be a consensus of what people actually want from sprite sheets. If reducing disk usage is a priority, your idea wouldn't cut it...

What I meant with "not mix and match" was that you intend to line up the frames like in jetrels example and not trying to squeeze in as many non-transparent-pixels into a an as little total area as possible (like was done in really old games to preserve diskspace).

/tsr - lurks away

I won't do a fixed-size rectangular grid since that would waste a lot of space, but I won't try either to make some images overlap on identical regions to save space, if that's what the old games in question did. (Unless I really have some extra time to kill, and develop or find an algorithm for that.)
There's a big screenshot on http://wiki.wesnoth.org/GSoC-SpriteSheets_Gabba that'll show you exactly the kind of sheet I want to build.
I updated that proposal with a better layout, btw.

This is an example of a halo (from Oracle.cfg).
(...)
My concern is that the halo images can be mixed and matched with other sprites, but not if both "halo" image & "image" image are part of the sprite on a single sprite-sheet image. Is that the case?

Well, if they end up on the same sprite sheet (which is not necessary or guaranteed, with what I'm proposing), this part of Oracle.cfg would like more or less like this:

Code: Select all

        [frame]
            duration=50
            image="some_sprite_sheet.png~CROP(0,0,10,10)"
            halo=some_sprite_sheet.png~CROP(20,20,15,15)
        [/frame]

This should work fine. We can also arrange for halos to be on their own sheet if needed.

[doofus-01]

We can also arrange for halos to be on their own sheet if needed.

That answers my concern. Thanks.

[Iris]

(The guy who implemented ~CROP() talking here, since that functor's being mentioned so much...)

One major concern I have with spritesheets at the moment is the way Wesnoth handles image caching. Although I had to work amidst the cache system for implementing ~CROP() and a bunch of other image functions back in the day, I never could completely understand how the cache mechanism works, but last night I wrote a tiny patch [1] for src/image.cpp to reveal very simple information about wesnoth's caching decisions with images that use ~CROP() or any other image path functions.

Taking one of my user-made campaigns, After the Storm as reference since it has the Time of Day pictures for the sidebar slapped together into a spritesheet, I observe that individual parts of the image are cached individually as follows:

Code: Select all

cache add: misc/schedule-twosuns.png !
cache add: misc/schedule-twosuns.png ! ~CROP(127,121,125,39)
cache hit: misc/schedule-twosuns.png !
cache add: misc/schedule-twosuns.png ! ~CROP(253,121,125,39)
cache hit: misc/schedule-twosuns.png !
cache add: misc/schedule-twosuns.png ! ~CROP(1,161,125,39)
cache hit: misc/schedule-twosuns.png !
cache add: misc/schedule-twosuns.png ! ~CROP(127,161,125,39)
cache hit: misc/schedule-twosuns.png !

As you can see, Wesnoth initially caches the whole spritesheet, and then it takes (as needed) every one of the parts retrieved after applying one or more image path functions — i.e. if we converted a single unit's animations into a spritesheet under this current mechanism right now, we'd be causing Wesnoth to consume more memory at run-time than it's worth for every single spritesheet.

I admit I didn't think of this when implementing the ~CROP() function but it wasn't my intention either to create a fully-featured spritesheet mechanism from there.

The problem appears to originate from the locator::load_image_sub_file() member method, which is in charge of applying transformations and invoking image functors. At the start of the method there's a call to get_image() which calls back to the method where we came from, except that this time it's to retrieve the target image file with no transformations.

[1]: http://shadowm.rewound.net/junk/wesnoth ... ching.diff

EDIT: minor rewording. Posting at 3:00 am turns out to be an error-prone action.

[Jetrel]

The one key thing wesnoth has that frogatto doesn't, which indeed was a great idea, is our progressive parameters. That's something frogatto doesn't have, and I might end up reimplementing it over there, since it was a really brilliant idea on your part.

more generally, frogatto's approch to describing animations is probably way too limited for wesnoth. It reduces animations to a sequence of images, but animations in wesnoth are much more than that...

sprite sliding is part of the animation engine, poison recoloring is poart of the animation engine, fade in/out is part of the animation engine, unit selection is part of the animation engine... and by "part of the animation engine" I mean that it's entirely possible to recode a special selection animation for a given unit instead of the default... so I wouldn't advise going the frogatto way for animation description.

That is not true. Pretty much everything you just described is in frogatto. The biggest difference is that instead of being in C++, we do all of that behaviour using the formula language. Everything from a unit's color, a unit's transparency, and a unit's position/rotation ... everything, are all variables you can modify in FML.

What we have are formulae that are executed on events, and in frogatto, events are fired at the beginning, end, and during every game cycle within an animation. So, let's take for example, poison animations. (frogatto doesn't have poison as a concept, but this exact behaviour is done with the color red for getting hurt).

on_poisoned="[animation('poisoned'),set(red, 100), set(blue, 100)]"
on_end_poisoned_anim="[animation('normal'),set(red, 255), set(blue, 255)]"

If we wanted to modify during every frame within the animation, we could set it to something like:
on_process_poisoned="[set(red, 100 * time_in_animation *some_formula), set(blue, 100 * time_in_animation *some_formula)]"

And no, we don't have to redo this for every unit in the game, we actually specify these in global prototypes that individual objects are derived from. So in the individual unit, we just specify the series of images used in the animation, and the behaviour is automatically applied to it, as long as we specified an object-prototype appropriate to it earlier in the file.




So this is a massive, different-to-the-core way of handling everything. Pretty much the entire game logic of frogatto is done in the formula language. Everything from object physics to AI. I just wanted to throw this out there as food for thought, as people are considering different implementations. I'm not suggesting doing stuff the frogatto way because that would require a ground-up rewrite of wesnoth. That'd be crazy. BUT - I want people to be aware of how it works, because just knowing what options are out there will make the design here, better.

(The guy who implemented ~CROP() talking here, since that functor's being mentioned so much...)

One major concern I have with spritesheets at the moment is the way Wesnoth handles image caching. Although I had to work amidst the cache system for implementing ~CROP() and a bunch of other image functions back in the day, I never could completely understand how the cache mechanism works, but today I have written a small patch [1] for src/image.cpp to reveal very simple information about wesnoth's caching decisions with images that use ~CROP() or any other image path functions.

Personally, I wouldn't do spritesheets until we reimplemented the graphics engine in OpenGL, purely because of the caching problems. If you think it's feasible to do by exposing the cache info, I'm not gonna stop anyone.

[fakedrake]

i took a look to the animation related code and had this idea on the subject:

We leave things pretty much be as they are in unit_animation.cpp/hpp and heavily edit the unit_frame class(and the related ones) so that instead of an individual image it contains simply the coordinates and the dimensions of the sprite and a pointer to the corresponding spritesheet wich we keep in the unit_animation class in a vector or sth. This way we can have animations which's frames spread across several files(even though we might not need it i thought it would be a cool trick )
Then we cut the sprite in display::render_unit_image() and blit it.
This very general plan is for C++ the code.

As for the WML there doesn need to be any change. Insted we can have a script in the image directory to instruct where to get each frame.

In other words the only thing we do is swap the surfaces for coordinates in images and put the corresponding instructions in separate script files...
This way the frames can be treated just like they were before and we can have spritesheets

I hope all that made sense...

[gabba]

(...)
Taking one of my user-made campaigns, After the Storm as reference since it has the Time of Day pictures for the sidebar slapped together into a spritesheet, I observe that individual parts of the image are cached individually as follows:

Code: Select all

cache add: misc/schedule-twosuns.png !
cache add: misc/schedule-twosuns.png ! ~CROP(127,121,125,39)
cache hit: misc/schedule-twosuns.png !
cache add: misc/schedule-twosuns.png ! ~CROP(253,121,125,39)
cache hit: misc/schedule-twosuns.png !
cache add: misc/schedule-twosuns.png ! ~CROP(1,161,125,39)
cache hit: misc/schedule-twosuns.png !
cache add: misc/schedule-twosuns.png ! ~CROP(127,161,125,39)
cache hit: misc/schedule-twosuns.png !

As you can see, Wesnoth initially caches the whole spritesheet, and then it takes (as needed) every one of the parts retrieved after applying one or more image path functions — i.e. if we converted a single unit's animations into a spritesheet under this current mechanism right now, we'd be causing Wesnoth to consume more memory at run-time than it's worth for every single spritesheet.
(...)

Hmm, that's very good to know. It makes sense that each part gets cached separately after applying functions, since other functors than ~CROP can actually change colors, etc, so the result needs to be cached individually.

I'll either:

1. make ~CROP not cache sub-images when it's used alone (or not cache sprite sheets, only the sub-images, depending on what works best); or maybe the solution could be not to cache sprite sheets at all, since they're a kind of pre-built image cache,
2. or if #1 doesn't make sense within the whole image functors framework, I'll write a different function that caches the sprite sheet properly, and loads sprites from it. I'll be reusing as much of ~CROP code as I can. A new WML syntax will be needed, but I'll keep it as simple as ~CROP, for instance:

Code: Select all

        [frame]
            duration=50
            image="{SHEET:spritesheets/saurians_sheet.png:(127,121,125,39)}units/saurians/oracle-magic-1.png"
            halo="{SHEET:spritesheets/halos_sheet.png:(1,161,125,39)}halo/saurian-magic-halo-1.png"
        [/frame]

The syntax is of course totally made up and I'll have to see what makes valid WML.
Anyways the interesting features in the WML above are:

• Fallback mechanism: if the "regular" png is not found, the game looks for the image in the sprite sheets. Not sure if fallback is a good feature or not, but it was suggested, so here's a syntax prototype for it. It does help finding out what the original image was, when debugging for instance.
• No new WML attribute is added, more simple and less error-prone IMO
• The part between { and } are meant to be automatically inserted during the build process, but you can still use them manually to support your hand-made sprite sheet. However for that the [frame_group] idea I introduced might be better...
• If for some reason you were already using ~CROP and a combination of other image functors (let's say you were using just part of a sprite and recoloring it), this will not add an additional ~CROP to the line, which could get confusing, even though it'd still work.

[Boucman]

FYI, the original reason the cache itself was inserted was that we used to have one disk access for each unit frame (getting the image from the disk every time) which is incredibly slow, esp on small device where the os won't cache all our data for us

however, it might be enough to cache the file and redo all the functors afterward, not sure how to mesure that...

another idea to throw around, rather than a new syntax, add a new functor ~SPRITESHEET() semantically similar to ~CROP() if, for some reason, we don't want to reuse ~CROP for that

[gabba]

I edited my proposal again to try and take into account the various things we discussed; I'm trying to adapt to all possible situations since opinions of how and whether to implement sprite sheets vary widely. In particular, I'll add support to my tool to put all animation frames from an existing unit onto a sprite sheet designed to be worked with by artists. Placement won't be perfect since frame sizes can vary, but unlike the optimized sprite sheets, frame ordering and important stuff like that will be preserved.
See as always http://wiki.wesnoth.org/GSoC-SpriteSheets_Gabba

Instead of re-reading all you can see a diff of all that was added to it since March 28, or just go to the page's history and check all non-minor revisions (those that don't have m in front).