Why A Manual Web Application Security Assessment is Worth Every Penny Over An Automated Tool
Choosing the Assessment: As a Security Consultant for SecureState, I have performed my share of Web Application Security Assessments in the last couple of years, including both Assessments which relied heavily on Web Application Scanners to find vulnerabilities in the web application, as well as Assessments which relied primarily on manual techniques. There is a great difference between these two techniques for performing Web Application Assessments, and I believe that the Web Application Scanner based Assessments are of little value compared to Web Application Assessments that rely on manual techniques. Many times, when choosing a Web Application Assessment, a company will choose the cheapest Assessment available without truly understanding the Pros and Cons of each Web Application Security Assessment. It is important to realize the fact that the cheapest option is not always the best option.
Tool Based Web Application Assessments: The first Assessment I wanted to address is a Web Application Scanner based Assessment. This Assessment heavily relies on Web Application Scanners in order to identify vulnerabilities in the web application. During this Assessment, a Web Application Vulnerability Scanner is run against the web application. Once the scan has finished running, a Consultant will manually verify the results of the scan and remove any false positives. This Assessment is about 80% Web Application Scanner based and only 20% manual based (the consultant generally validates the vulnerability scanner findings). This Assessment is cheap because it requires little manual effort. The primary danger with this Web Application Assessment is the fact that many organizations do not realize the limitations of most Web Application Scanners. Once an organization has a Web Application Scanner based Assessment performed and only a few vulnerabilities are identified, the organization may falsely believe that their web application is secure.
Manual Web Application Assessments: The second Web Application Assessment I will be talking about primarily uses manual techniques in order to identify web application vulnerabilities. In most cases, the Consultant will use web application proxies which sit between the web application and the Consultant’s browser. This is done in order to bypass client-side constraints. The bypass of client-side constraints aids the Consultant in identifying vulnerabilities which would have been missed if client-side validation was used. Before the Web Application Assessment starts, the Consultant is normally given information about the business logic behind the web application. Most of the time, manual Assessments are performed using credentials of a privileged user. This is done in order to find vulnerabilities an authenticated user may be able to exploit (or fall victim to) that a non-authenticated user cannot. If the web application uses different privileged accounts, then the Consultant will try and use the lower privilege account to access functions the higher privilege account is able to access. The Consultant will also try to obtain information from accounts with the same privilege level. For example, if Account A is an administrator and Account B is a regular user, the Consultant will attempt to use Account B to access Account A’s data. Once manual testing of the web application is completed, the Consultant normally runs a Web Application Vulnerability Scanner as a sort of sanity check. This is performed in order to verify the Manual Assessment did not miss any vulnerabilities which could be considered low hanging fruit. Most of the time, Manual Web Application Security Assessments are about 80% manual and only 20% Web Application Scanner based.
Basic Logic Flaws: Web Application Scanners do not have the ability to think (unless we really are living in the “Matrix”) and are therefore limited in their ability to identify flaws in a web application’s logic. Let us look at a real life example of a logic flaw in a web application. Suppose a Website is created for a person to order expensive jewelry online. For the purposes of our example, the name of the site with the expensive jewelry for sale is https://BillionDollarJewelry.com. On this site, items are added to the user’s cart using a URL which accepts two parameters. The two parameters in our example are “Item”, which contains the value of the item being added to the cart, and the parameter “Cost”, which holds the cost of the item being added to the cart. In our example, the URL used to add items to a customer’s cart is as follows:
In this URL, the BigDiamondRing item is added to the cart with a cost of 1000000000 dollars. The problem occurs when an attacker is able to purchase the BigDiamondRing for the on sale price of just zero dollars by sending the following URL:
As far as I have seen, Web Application Scanners are not able to identify these sorts of vulnerabilities, yet I have found them numerous times while manually reviewing a web application.
Basic Privilege Escalation Flaws: I have never encountered a Web Application Scanner with the ability to identify privilege escalation vulnerabilities. In most Manual Web Application Assessments, the Consultant will attempt to escalate privileges using accounts with different levels of access, and privilege escalation vulnerabilities are common to find. For example, suppose User A is part of the web application’s administrator role. When User A logs into the web application, they are presented with a list of links which direct the user to web pages with powerful functions. In our example, we will call these functions X, Y, and Z. User B comes along and is only a part of the basic user role. Upon logging into the web application, he is presented with links which direct User B to web pages for functions A, B, and C. If the web pages which provide functions X, Y, and Z are not directly tied to the Administrator role, then User B may be able to access functions X, Y, and Z by directly visiting the function’s web page.
Other Common Vulnerabilities (XSS): It is also common to find high profile vulnerabilities which many Web Application Scanners miss while performing Manual Web Application Security Assessments. For example, I have seen a form of cross site scripting which requires a series of steps in order to get the malicious script to execute. If this series of steps is altered in any way, the script will not execute and the cross site scripting is not identified. Suppose a web application brings a person through a series of steps which require a customer’s personal information to be inserted into the web application. Once all data is placed in the web application, a summary of all data placed into the web application is shown to the customer. For our example, Web Page A requires the customer to submit their first and last name, Web Page B requires the customer to insert their address, Web Page C requires the customer to insert their phone number, and Page D shows the customer a summary of the information inserted into the web application. Now if the FirstName parameter on Page A is given a value of “Tom”><script>alert(‘xss’)</script>” the malicious script will not execute until Page D loads. Vulnerabilities like these are frequently found in the course of a Manual Web Application Assessment, but Web Application Scanners rarely find these sorts of vulnerabilities.
Other Common Vulnerabilities (SQL Injection): I have also encountered a number of SQL Injection vulnerabilities which many Web Application Vulnerability Scanners miss, but that manual review of the web application will identify. Many Web Application Scanners miss Blind SQL Injection in non-common databases. For example, I have found Blind SQL Injection in web applications using a DB2 database. The Web Application Scanner did not identify this vulnerability. I manually used DB2 string concatenation techniques in order to identify and exploit this vulnerability. A breakdown of this SQL Injection is as follows:
- Original Query: https:DemoUserSite.com/LookUpCustomerID.aspx?UserId=Timmy (This page returns Timmy’s information)
- Modified Query 1: https:DemoUserSite.com/LookUpCustomerID.aspx?UserId=Tim’my (This page returns no individual’s information, or some default page)
- Modified Query 2: https:DemoUserSite.com/LookUpCustomerID.aspx?UserId=Tim’ || ‘my (This page returns Timmy’s information)
Summary: Although Web Application Scanner based Assessments are cheaper than Manual Web Application Security Assessments, they can miss many vulnerabilities which a manual Assessment will identify. Many times Web Application Scanner based Assessments will miss flaws in a web application’s logic, role segmentation vulnerabilities, and other non-standard common vulnerabilities. Manual Web Application Assessments may be more expensive, but when it comes to protecting sensitive information they are much more effective than Assessments which primarily rely and on Web Application Scanners.