Skip to content

To limit noise_shape in func keras.layers.Dropout() #21768

@ILCSFNO

Description

@ILCSFNO

Bug Issue

The doc of keras.layers.Dropout() shows its description as below:

noise_shape: 1D integer tensor representing the shape of the
binary dropout mask that will be multiplied with the input.
For instance, if your inputs have shape
`(batch_size, timesteps, features)` and
you want the dropout mask to be the same for all timesteps,
you can use `noise_shape=(batch_size, 1, features)`.

For the repro below, which is expected to raise error, I found that noise_shape in keras.layers.Dropout can be any value without raising any error, including the negative numbers.

Using tf 2.20.0 and keras latest:

Repro

import tensorflow as tf
import keras
noise_shape = (1000, 2124, 6415, 461, -45616, 61515, -100, 123)
input_data = tf.random.normal((10, 5))
model = keras.models.Sequential([keras.layers.Dense(64, activation='relu', input_shape=(5,)), keras.layers.Dropout(rate=0.5, noise_shape=noise_shape), keras.layers.Dense(10, activation='softmax')])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.summary()
output = model(input_data)
print(input_data.shape, output.shape)

Output

Model: "sequential_10"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ dense_24 (Dense)                │ (None, 64)             │           384 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_17 (Dropout)            │ (None, 64)             │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_25 (Dense)                │ (None, 10)             │           650 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 1,034 (4.04 KB)
Trainable params: 1,034 (4.04 KB)
Non-trainable params: 0 (0.00 B)
(10, 5) (10, 10)

I think that noise_shape should be limited, but not sure if this is expected actually.

Thanks for noting!

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions