Methodology

Research Methodology

Overview

The goal of this report is single: to help organizations choose the most appropriate vendor for their environment. To do so, the following guidelines were used:

DDoS-Intro-Image

  • Technical focus
    The analysis is focused on a technical analysis rather than a business one.
  • DDoS features only
    All vendor reviews here provide much more than DDoS: CDN, WAF, load-balancing and more. This repot focuses only on DDoS.
  • War-time evaluation
    There is a great emphasis on the services’ performance under attack, in their money-time, in addition to daily peace usage.
  • Cooperate with vendors
    Reviewed vendors were approached. They were asked for information and evolution licenses, and were also asked to comment on the report prior to its publication.
  • Source of information is public documents, hands-on and vendors
    The report is based on public documents, hands-on experience with the products and feedback received and verified by vendors. DDoS or any lab testing was not included.
  • Break-down method: (1) Deployment, (2) Mitigation, (3) UX & Reporting, (4) Stability & Support and (5) Pricing: This break-down method was used to analyze each vendor. A scoring system was developed for each section to score each vendor.

Cooperation With Vendors

In this report, interaction with the vendor plays a great role. Each vendor was asked to provide a focal point to collect technical and business materials and to answer inquiries. In addition, an evaluation, or at least a demo, was requested.

Incapsula and F5 cooperated with our research, while CloudFlare did not. This is the reason for some missing aspects regarding CloudFlare’s analysis.

 

Proxy / Caching

In DDoS mitigation, web caching is done by reverse-proxies which act in tandem as prominent line of defense by blocking attacks from reaching the web server. (read more)

Public materials

Vendor cooperation

Demo

Hands-on

Level of Cooperation With Each Vendor

Technical Focus

The analysis is focused on a technical analysis rather than a business one.
This is not the first DDoS report out there. The Forrest’s DDoS Services Providers, Q3 2015 (or obtain it free of charge here) is methodological and worth reading. However, it takes a ten-thousand-foot view and presents more business-oriented aspects, like market size and global presence. While these factors are important in the vendor selection process, our approach is a more technical. In particular, we examine deployment factors and mitigation factors. Another competitive analysis is Top-10 DDoS Protection Services Reviews. This analysis provides a good introduction for beginners; however, the analysis itself is very flat and includes no DDoS features.

DDoS Features Only

Only DDoS features are reviewed; CDN and generic WAF are excluded.
The report reviews only DDoS mitigation capabilities. Although some of the vendors offer an internet acceleration Content Delivery Network (CDN), Web Application Firewall (WAF) or other interesting technologies, they are all disregarded unless they have any DDoS mitigation value. In reality, organizations may add no DDoS-related aspects to their overall decision.

War-Time Evaluation Focus

Products will be evaluated here with a greater focus on how they perform under attack than in peace time.
Perhaps the biggest problem with DDoS is that peace time can last as long as one to two years, creating the sensation that everything works well.  This report is mostly concerned with how thing will work in war time. Will the attacks be blocked? What kind of visibility and control you will get? Will there be false positives?

Source of Information

Data is based on vendors’ public materials, discussion with vendors and a user interface review.
The report does not include testing and/or the reputation of the vendor. As mentioned above, for vendors that have fully interacted with the research, a detailed analysis is presented as is a competitive analysis. For the rest, only a basic analysis is provided.

Additional Methodology Used

Weight-Based Evaluation

The report a priori assigns weights to different features based on their estimated value to customers.
There are literally hundreds of features that can be reviewed in DDoS mitigation. We have a priori selected the features we consider most important and have assigne a weitght to each. Our weight system is based on the weights we think customers should assign to each feature.

The weight-based system gives our analysis two advantages. The weight system forces the evalution to focus on the important features on which we decided a priori. For example, the branding of the vendor has no importance because branding was decided a priori to not be a factor.

The weight system also boosts the objectivity of the report. With it, the review becomes a technical job of marking each vendor according to which features exist (and to what extent) and which do not.

Note that some aspects of the service that we considered important are missing simply because we were not able to objectively measure them. This includes the stability of the service and the support level. In many cases, we did not have access to pricing. The aspects we were not able to cover are stated passed for the organization to complete, as indicated in the Next Steps – Completing your Evaluation section; we still plan to cover them in the future.

Existing Features Only

The analysis ignores the vendor’s roadmap; only existing features are evaluated.
The organization’s roadmap is not included in the analysis. This report evaluates only what is out there at the time of analysis. It is planned to update the report on a regular basis.

Break-Down Method

There are so many aspects of DDoS solutions. To create order in this domain, the analysis is divided into five parts.

We were able to cover the first three rather well. Stability & Support has not been covered well so far. (We are planning to complete this in the future.) Pricing was covered partially because not all vendors provided it.                           squar

Feedback

Feedback on this report is welcome and should be sent to [email protected]


  

  

Newsletter

Stay up to day with the latest DDoS news

Error: Contact form not found.