Reports

after asking around my question on Image.sc forum, I found that the problem can be solved if I downgrade keras from 3.6.0 to 3.4.1 (Google Colab version).

However, the bad news is that a new bug appeared. When I set

discriminator.compile(loss = "binary_crossentropy", optimizer = Adam(learning_rate = 0.0002, beta_1 = 0.5), metrics = ["accuracy"])
# Set non-trainable after compiling discriminator
discriminator.trainable = False

discriminator still becomes non-trainable even if it had been compiled before turning off the trainable tag, and no re-compilation of the discriminator afterwards. Please refer to discriminator.summary() before and after turning off the trainable flag.

discriminator.summary before freezing
Model: "functional_1"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Layer (type)                         ┃ Output Shape                ┃         Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ input_layer_1 (InputLayer)           │ (None, 28, 28, 1)           │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ flatten (Flatten)                    │ (None, 784)                 │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_4 (Dense)                      │ (None, 512)                 │         401,920 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ leaky_re_lu_3 (LeakyReLU)            │ (None, 512)                 │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_5 (Dense)                      │ (None, 256)                 │         131,328 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ leaky_re_lu_4 (LeakyReLU)            │ (None, 256)                 │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_6 (Dense)                      │ (None, 1)                   │             257 │
└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
 Total params: 533,505 (2.04 MB)
 Trainable params: 533,505 (2.04 MB)
 Non-trainable params: 0 (0.00 B)
discriminator.summary after freezing but no recompile
Model: "functional_1"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Layer (type)                         ┃ Output Shape                ┃         Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ input_layer_1 (InputLayer)           │ (None, 28, 28, 1)           │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ flatten (Flatten)                    │ (None, 784)                 │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_4 (Dense)                      │ (None, 512)                 │         401,920 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ leaky_re_lu_3 (LeakyReLU)            │ (None, 512)                 │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_5 (Dense)                      │ (None, 256)                 │         131,328 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ leaky_re_lu_4 (LeakyReLU)            │ (None, 256)                 │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_6 (Dense)                      │ (None, 1)                   │             257 │
└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
 Total params: 533,505 (2.04 MB)
 Trainable params: 0 (0.00 B)
 Non-trainable params: 533,505 (2.04 MB)

There are a lot of reports about this tag working aberrantly A GitHub discussion Another GitHub discussion To circumvent this problem, I have to turn discriminator trainable before I train it and then off before I train the entire GAN in each epoch cycle

discriminator.trainable = True
discriminator.train_on_batch(x = realhalf, y = np.ones((half_size, 1)))
discriminator.trainable = False
combined.train_on_batch(np.random.normal(0, 1, (batch_size, 100)), np.array([1] * batch_size))

But honestly I feel a bit unsecure using these packages since the bug has been there since almost a decade ago. I did a brief search and some recommended PyTorch instead (Redit discussion)? If anyone knows about these packages, please feel free to let me know. Thanks in advance!

79174641