-
Couldn't load subscription status.
- Fork 19.6k
Open
Labels
Description
Bug Issue
The doc of keras.layers.Dropout() shows its description as below:
keras/keras/src/layers/regularization/dropout.py
Lines 27 to 32 in 465a56d
| noise_shape: 1D integer tensor representing the shape of the | |
| binary dropout mask that will be multiplied with the input. | |
| For instance, if your inputs have shape | |
| `(batch_size, timesteps, features)` and | |
| you want the dropout mask to be the same for all timesteps, | |
| you can use `noise_shape=(batch_size, 1, features)`. |
For the repro below, which is expected to raise error, I found that noise_shape in keras.layers.Dropout can be any value without raising any error, including the negative numbers.
Using tf 2.20.0 and keras latest:
Repro
import tensorflow as tf
import keras
noise_shape = (1000, 2124, 6415, 461, -45616, 61515, -100, 123)
input_data = tf.random.normal((10, 5))
model = keras.models.Sequential([keras.layers.Dense(64, activation='relu', input_shape=(5,)), keras.layers.Dropout(rate=0.5, noise_shape=noise_shape), keras.layers.Dense(10, activation='softmax')])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.summary()
output = model(input_data)
print(input_data.shape, output.shape)Output
Model: "sequential_10"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type) ┃ Output Shape ┃ Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ dense_24 (Dense) │ (None, 64) │ 384 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_17 (Dropout) │ (None, 64) │ 0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_25 (Dense) │ (None, 10) │ 650 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 1,034 (4.04 KB)
Trainable params: 1,034 (4.04 KB)
Non-trainable params: 0 (0.00 B)
(10, 5) (10, 10)
I think that noise_shape should be limited, but not sure if this is expected actually.
Thanks for noting!