The problem:
I have thousands of images for my game, how do I store and load them?

A bad solution:
Create a folder called 'resources' and drop them all in there, name them accordingly: blue_robe_001.png to blue_robe_550.png purple_robe_001.png to purple_robe_550.png Okay well thats 1,100 images for 2 assets, there is 3 robes and 3 staffs to accompany them, so 3,300 assets just for the player and equipment. Not a problem, we all have large hard drives now adays? Well the size isn't the issue, the issue is the quantity of images.

I also need background scenery, game play objects, entities, sounds, etc. I would estimate over 50,000 assets by the projects completion and that would make a supercomputer cry having to index and allocate all these files to the drives file system.

A better solution:
Sprite sheets! Game developers have been using sprite sheets for decades! Position all the sprites on a single image and just draw the section required, that’s a lot faster and I only need to load one image into memory so that would be super fast too…

Well, maybe. When exporting the sprites they were in a 500x500 frame, so assuming 22 sprites per row and 25 rows, I would require 11000x12500 pixel image… Okay so not amazing, but once in memory I just need to draw the section I need… Just out of curiosity, how big would that file be?

I Fired these numbers into ChatGPT to give me an estimate:

Right... Okay something I should mention here is that we are limited to a 500MB project size for submission...

 - "I submit to you, Legend of the blue guy on a white screen"

Okay so I've waffled on enough to help the character count, lets got down to my solution.

I need to host thousands of images which are quite large, I need to store them in a minimal quantity, and I need the file size to be small and ultimately, I need to be able to load, unload, and draw quickly.

So I have my sprites, thats a given, but like i said I exported them out at 500x500 pixels, however, there are empty rows at the bottom of the image and empty columns to the right, I can easily resize these images to remove them, thats a good saving!

Got some empty space at the left and top of the sprite too, issue is, if I remove them, they wont animate correctly, but thats not a problem, I just need to keep track of how many rows (Y Offset) and columns (X Offset) per image, the savings from the image data greatly outweight the cost of storing 2 integers.

So now i have a collection of trimmed sprites and some meta data, so lets find a way to store it.

Just stack them ontop of each other? Sounds good to me!

I have spoken about my Library editor and file format in another post, so I'll just skip past that and show the results:

Starting:
Files: 550
Combined size: 238MiB

Ending:
Files: 2
Size: 9.12MiB

Massive improvement! Now, let's get this working in C++

Since I wrote the file compiler in C# I didn't want to write objects directly, so I made sure to write each part as something I could easily read in C++:

Byte: Version (1)
Char[]: Header (Sky Wizard Asset Library)
Byte: Library Type (1)
int: Entry count

For each entry:

int: X Offset
int: Y Offset
int: Image data length
Char[]: Image data

Not wanting to get AI to translate my C# code into C++ I did a bit of research but what came up was always writing in C++ then reading it back... But thats not a problem, I can just cherrypick the lines of code which would be applicable to my use case.

I found this article by Conifer Productions (reading-binary-files-cpp) which had pretty much all the information I needed, so I went ahead and wrote a reader and to my suprise I managed to get everything loaded into memory correctly first time!
Well, almost. The code I wrote was correct, my validation of reading it had one issue, the Header (Sky Wizard Asset Library).
I read the bytes correctly but being unfammiliar with strings in C++ I didn't realise I had to add a null byte at the end, but this was quickly resolved by comparing my string to the one declared in code by comparing the bytes.

Coming from C# to C++ as a mature student has open my eyes to how much hand-holding C# does for the developer.

At this point I would like to say that I didn't believe this would be the final form of the Texture Library, I had envisioned a streamable library that could be read and wrote to at the same time, there would be a FAT and Meta data held together and the texture data would be smashed sequentially with no other data between. The plan was to load all the image data at once as block of data and then use pointer arithmatic to quickly navigate to the point in memory needed. But considering it takes about 0.2 seconds to load and iterate through each sprite and creating objects one by one, I don't think that this is a time saving exorcise I need to undergo presently.

So now comes the fun part. Making the code good.

I think it is safe to say that alot of us when programming start off by writing bad code, I don't mean we are bad at programming, I mean we just slap the keyboard until we get something that works, just to make sure the idea does work as intended and there are no major unforseen issues, then once we are satisfied our ideas are feasable, we make the code better.

So why is my current Texture Library class bad?

Well, it loads the whole library up on startup, even if the assets are not being used.

My first solution to this was to create another constructor which took in a vector of integers called 'indices'. If the indices where empty, load the whole library, if there was values, only load those entries. That way I could just move the loading code into the new constructor and get the current constructor to call that with an empty indicies vector.

That worked fine, in theory, but it means at point of initialization i would either have to know which indices I needed or I would be loading the whole library up again. It also meant that if I wanted to load up additional indicies I would need to dispose the object and recreate. Not idea.

I went through a few more iterations of how to achieve a good implimentation of what I was looking for until I settled on what I have now.

I decided to create entries in the asset manager for every library within the resources folder, this meant I had the correct array lengh, offsets and I even decided to add the starting position of each image data into an array too, sure it is a few more cycles at the start of the application but thats fine, a quick loading screen isn't going to be too painful for the player.
Having the position of the data in memory acts just like the FAT within the file type I was after, so I didn't need to iterate through the whole library to get some images at the end, I knew the offset, I could just use seekg() and start reading! Absolutely beautiful!

I moved the actual code to read the library into it's own function with the indicies as an input so it could be reused with ease.

This whole refactoring took me about 8 hours of trial and error to see what would work and what wouldn't and it has left me with one function left to create, UnloadIndices.

That should be simple enough, from LoadIndices, I can pass through the std::vector<int> &indices and the UnloadIndices could work by iterating through the array and disposing everything not in the indices vector.

https://github.com/PaulMcGinley/SkyWizards/commit/9fc4fb9d5a646f411b6dea30388b874cc2e3ff61#diff-2d029117ffa3d6961c32ea7c7cf9831c4629cb2b4f07a5c23b9dcde57dd463d3