Tensorflow:your Input Ran Out Of Data When Using Custom Generator

I am using custom generator to pass my data. But i keep encountering an error which says i have run out of data and to use repeat() when passing the dataset. i am using plain gener

Solution 1:

If you aren't using repeat (and even if you are it's good for debugging) the first thing you need to check is how many elements your generator creates. a quick way to do that would be with something like

len([g for g in generator(idir,odir,4,True)])

Then you need to make sure that your steps per epoch times your batch size is less than that number.

You can use repeat even with that generator though - you just need to wrap it with a tf.dataset like this:

gen = lambda : generator(idir,odir,4,True)
dataset = tf.data.Dataset.from_generator(gen, output_types=(<types>), output_shapes=(<shapes>)).repeat()

you have to specify output type and shape but then it works fine.