Skip to content

Scaling dropout layer by keep probability during test time - #106

Closed
Chris-Nicholls wants to merge 1 commit into
karpathy:masterfrom
Chris-Nicholls:bugfix/keep_prob
Closed

Scaling dropout layer by keep probability during test time#106
Chris-Nicholls wants to merge 1 commit into
karpathy:masterfrom
Chris-Nicholls:bugfix/keep_prob

Conversation

@Chris-Nicholls

Copy link
Copy Markdown

From the dropout paper http://www.cs.toronto.edu/~rsalakhu/papers/srivastava14a.pdf :

If a unit is retained with probability p during training, the outgoing weights of that unit are multiplied by p at test time

The test activations should be scaled by (1-drop_prob), not drop_prob.
For example, if drop prob is 0, this layer should have no effect and we should scale activations by 1.

@ratajczak

Copy link
Copy Markdown

Hi Chris, it looks like duplicate of #61

@Chris-Nicholls

Copy link
Copy Markdown
Author

Yup, you're right. Looks like this isn't being maintained anyway.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants