Skip to content

Rate-limiter in email send causes high-latency alerts #2688

Description

@bboreham

The rate-limiter seems to be doing its job, to slow down people trying to spam the sign-in page.
However this then shows up as a high-latency alert, e.g. on 2020-06-27:

2:46 AM
Weave CloudAPP Instance: Weave Cloud (prod)
(UsersLatency - WARNING FIRING) default/users: high latency
Impact: Users have a very slow experience logging in
99th percentile latency: 1.650
View firing alerts Playbook Dashboard
containerName: users
description: The users service has a 99th-quantile latency of more than 0.5 seconds for 5m.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    chorerefinement/improvement of end user functionality; making things work bettermonitoringChanges to how we monitor Weave Cloud itself

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions