![]() Here it’s locating our image names, put them in a list and shuffle it. _init_ is the initialising method, it’s called when the class is instantiated.The class needs a few methods in order to function correctly: We put as arguments important information such as the directory containing the data (data_dir), the batch size, size the images will be rescaled to (for this purpose they’ll have the same height and width), number of images to use (setting this to a number less than the total number of images is helpful for testing the network and debugging), and whether the data should be shuffled each epoch. You’ll notice I’m calling a function ‘augment’ in this code, you can find the code for that here, or make your own function where the input is an image, and the output is an augmented version of that image, with fixed size (im_size), scaled between -1 and 1. Sequence, so that I can capitalise on perks like multiprocessing. Tensorflow data generatorįor my Tensorflow data generator, I’m going to inherit from tf. This can lead to huge efficiencies during training, since it allows for data to be prepped on the CPU which the GPU is running training. A data generator is a great option which allows you to generate the data in real time, run preprocessing and augmentation in batches, and feed it right in to the model. But datasets too large to load in to memory are becoming more common, so it’s important to have a pipeline that can deal with those situations. My dataset isn’t very large (25,000 fairly small images, of which I’ll only be using 1000 as a minimal example), so I can load it all in to memory. My dataset is stored in a subdirectory (‘data’) of folder containing my training script (‘folder’), with the following structure: folder/ ├── data/ ├── test/ ├──1.jpg ├──. If you want to follow along exactly, download the data from here. Mostly though, I’m using this dataset because if I’m going to spend vast amounts of time looking at images I’d rather they be of cute animals. I’ll be using a familiar dataset, Cats v Dogs, because this guide isn’t about solving a tricky problem (you’ve probably already got your own problem in mind), it’s about creating a general, minimal example that you can easily adapt. Photo by Chris Arthur-Collins on Unsplash Dataset
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |