Denial of Service (DoS) : Some Thoughts
About a year ago, I had the opportunity to solve a class of Denial-of-Service attacks that were compromising our availability and scalability. During that investigation, I happened upon a revelation. That revelation led to a solution. I’ve since seen that learning applied to other systems, including Amazon’s SimpleDB, so I wanted to share it here.
Consider the following scenario (also depicted below):
- A web client issues an HTTP request to a web site
- The web site, upon receiving the request, attempts to determine if the current request is part of a larger DOS attack
- If so, a defense is executed
- If not, the web request follows a normal execution of business logic
- The web server returns a response to the web client
It is relatively simple and inexpensive for a client to issue an HTTP request. The server on the other hand needs to do a lot more work, which may include:
- Reading data from data bases
- Running algorithms and computations on that data
- Filling caches
- Creating an HTTP response
These are more expensive than simply issuing an HTTP request - i.e. what the client does. While a server is fulfilling a client’s request, any of the following could be considered as reserved for that request:
- A front-end server connection
- A fraction of a back-end (e.g. DB) connection if connections are pooled
- A fraction of a middle-tier (e.g. an internal web service) connection if connections are pooled
- CPU and memory resources on the front-end, middle-tier, and back-end servers
All of the resources above are in limited supply, so at any point of time, a web site can support a finite number of client requests.
To DoS a site, all a client needs to do is to overrun that limit.
How to DoS a Site?
Find an operation on a website that typically takes a lot of time and bang on it with Curl or wget. For example, search or login might be slow operations as they require a lot of work on the back-end. (Disclaimer: I am not suggesting that anyone DoS any site.)
The principle behind DoS is very simple:
Have the client do much less work than the server. Hence, a laptop in Starbucks can easily issue 4k HTTP requests per second. If those requests are expensive for a website to serve, 4K of those requests may overwhelm it.
DoS Defense
How do you defend against DoS? Follow these rules:
- Ensure that your DoS detection logic is run as early on as possible in your server’s request processing stack
- Ensure that all resources (except for front-end server connections) are O(1) with respect to request traffic under a DoS attack.
- This means that under attack conditions, web site resource consumption does not increase — not memory, not CPU, not back-end server connections, nothing. Your inbound client connections a.k.a. your front-end server connections can increase — that’s OK.
- If you detect a DoS attack, return the cheapest response possible as soon as possible: e.g. HTTP 503 - Server Unavailable.
The basic principle behind fending of a DoS attack is to make responding to attacking requests as cheap as possible so that the server or server farm can keep up with the client.
My DoS Defense
To solve this class of DoS attacks, I ran a simple algorithm to detect the DoS attack. I made sure that the detection algorithm did not create connections or eat up memory or CPU — in effect, these resources were O(1) or constant. This meant relying on local data.
If I detected a DoS attack, I cached some client information in a local cache. This caused subsequent requests to be locked out — a fast failure response was sent to the client.
-
rooksfury posted this
nosql
sharadag
boredandroid