02_firmware_analysis_management

Chapter 2: Firmware Analysis Management

In Chapter 1: User Authentication & Authorization, we learned how EMBArk ensures that only the right people can access its powerful tools. Now that we know who can use the system, let's explore what they do with it: managing the firmware analysis itself.

Imagine you're an analyst with a new firmware from a device. You need to upload this firmware, tell EMBArk exactly what kind of security checks to perform (e.g., look for specific vulnerabilities, analyze its file system, or check for weak passwords), and then kick off the analysis. While the analysis is running, you want EMBArk to keep track of its progress, and once it's done, store all the results neatly.

This entire process, from getting your firmware into EMBArk to applying specific analysis rules, initiating the scan, and keeping a detailed record, is handled by "Firmware Analysis Management". Think of it as the central control tower for all firmware evaluations, directing each analysis through its journey from intake to final review.

Solving Our Use Case: Uploading Firmware and Starting an Analysis

Let's walk through how you would use EMBArk to upload a firmware file and start an analysis with custom settings.

1. Uploading the Firmware File

The first step is to get your firmware file into EMBArk. You'll typically do this through the "Uploader" section of the web interface or via an API.

How it works (Web Interface): You would navigate to the "Uploader" page within EMBArk (e.g., embark-uploader-home). On this page, you'll find an option to select and upload your firmware file.

When you upload a file, EMBArk saves it securely and creates a unique record for it in its database.

# Simplified snippet from embark/uploader/views.py
from uploader.models import FirmwareFile
from django.core.files.uploadedfile import UploadedFile # Represents an uploaded file

def save_file(request):
    # This function is triggered when you upload a file.
    for uploaded_file in request.FILES.getlist('file'):
        # 1. Create a new entry in the database for the firmware file.
        firmware_file_record = FirmwareFile.objects.create(file=uploaded_file)
        # 2. Link the file to the user who uploaded it.
        firmware_file_record.user = request.user
        firmware_file_record.save()
        # ... (display a success message to the user) ...
    return HttpResponse("Upload successful") # A simple web response

This code snippet shows how a FirmwareFile object is created and saved, ensuring each uploaded file is tracked and associated with its owner.

2. Defining Analysis Settings

After your file is uploaded, you need to tell EMBArk how to analyze it. This is done through an "analysis form" where you specify various settings.

The Analysis Form: On the uploader page, you'll see a form that allows you to customize your scan:

<!-- Simplified snippet from embark/templates/uploader/start.html -->
<form action="{% url 'embark-uploader-start-analysis' %}" method="post" id="analyze-form">
    {% csrf_token %}
    <label for="SBOMModeSwitch">SBOM mode</label>
    <label class="switch">
        <input id="SBOMModeSwitch" type="checkbox" onclick="SBOMModeToggle()"/>
        <span class="slider round"></span>
    </label>
    <br>
    <label for="expertModeSwitch">Expert mode</label>
    <label class="switch">
        <input id="expertModeSwitch" type="checkbox" onclick="expertModeOn()"/>
        <span class="slider round"></span>
    </label>

    <!-- Fields for firmware version, notes, device, architecture, emulation, etc. -->
    {% for field in analysis_form %}
        <!-- Example: A basic field for firmware version -->
        {% if field.name == "version" %}
            <div class="row">
                <label for="id_version">Firmware version</label>
                <input type="text" name="version" id="id_version">
            </div>
        {% endif %}
        <!-- Example: A field for scan modules, typically shown in 'Expert mode' -->
        {% if field.name == "scan_modules" %}
            <div class="row expertModeOptions" value="expmode_off">
                <label for="id_scan_modules">Scan modules</label>
                <!-- Checkboxes to select specific analysis tools, like 'F02_toolchain' -->
            </div>
        {% endif %}
    {% endfor %}
    <button type="submit">Analyze</button>
</form>

This form, built using FirmwareAnalysisForm (from uploader/forms.py), lets you configure:

• Basic Details: Such as the version of the firmware and any notes you have about it.
• Associated Devices: You can link the firmware to specific devices or platforms using the device field. This uses other simple forms (DeviceForm, VendorForm, LabelForm) to help you manage device information.
• Expert Mode Options: By enabling "Expert mode," you unlock advanced settings. These include specifying the firmware_Architecture (e.g., ARM, MIPS, x86), enabling user_emulation_test or system_emulation_test to run the firmware in a simulated environment, or selecting individual scan_modules.
  • Scan Modules: These are specific analysis tools or checks that EMBA (EMBark's backend analyzer) can run. For example, F02_toolchain identifies the software building tools used for the firmware, while S10_binaries_basic_check performs fundamental checks on executable files within the firmware.

3. Initiating the EMBA Scan

Once you've configured your settings and clicked "Analyze," EMBArk takes over to start the scan.

The start_analysis View: This function in EMBArk's code processes your form submission and begins the orchestration of the EMBA scan.

# Simplified snippet from embark/uploader/views.py
from uploader.forms import FirmwareAnalysisForm
from uploader.models import FirmwareAnalysis, FirmwareFile
from uploader.executor import submit_firmware

def start_analysis(request):
    if request.method == 'POST':
        form = FirmwareAnalysisForm(request.POST)
        if form.is_valid():
            # 1. Save the analysis settings from the form to a new FirmwareAnalysis record.
            new_analysis = form.save(commit=False) # Prepare, but don't save yet
            new_firmware_file = FirmwareFile.objects.get(id=new_analysis.firmware.id)
            new_analysis.user = new_firmware_file.user
            new_analysis.firmware_name = os.path.basename(new_firmware_file.file.name)

            # 2. Add any associated labels from the selected devices.
            for device in form.cleaned_data["device"]:
                if device.device_label:
                    new_analysis.label.add(device.device_label)
            new_analysis.save() # Now save the complete FirmwareAnalysis record

            # 3. Hand over the new analysis and firmware file to the executor to start the scan.
            if submit_firmware(firmware_analysis=new_analysis, firmware_file=new_firmware_file):
                return redirect('embark-dashboard-service') # Go to dashboard to see progress
            # ... (handles errors if submission fails, e.g., queue full) ...
        # ... (handles cases where the form is invalid) ...

This function creates a new FirmwareAnalysis record in the database, linking it to your uploaded FirmwareFile and all the settings you chose. Then, it calls submit_firmware, which is the key function that prepares and starts the actual EMBA analysis.

4. Tracking Analysis Progress (Quick Note)

After initiating the scan, you'll be redirected to the EMBArk dashboard. This is where you can monitor the real-time progress of your analysis, a topic we'll explore in detail in Chapter 3: Real-time Progress Monitoring.

Under the Hood: The Air Traffic Controller in Action

Let's take a simplified look at what happens behind the scenes when EMBArk manages a firmware analysis.

The Analysis Journey: A Simple Sequence

When you click "Analyze," here's a simplified sequence of how EMBArk directs the process:

sequenceDiagram
    participant User
    participant EMBArk Web Server
    participant Database
    participant EMBArk Executor
    participant EMBA Analyzer

    User->>EMBArk Web Server: Submits analysis form
    EMBArk Web Server->>Database: Creates FirmwareAnalysis record
    Database-->>EMBArk Web Server: Analysis ID confirmed
    EMBArk Web Server->>EMBArk Executor: Calls submit_firmware (Analysis ID, Firmware File)
    Note over EMBArk Executor: Prepares analysis environment and EMBA command
    EMBArk Executor->>EMBA Analyzer: Runs EMBA scan with specific flags
    EMBA Analyzer-->>EMBArk Executor: Generates logs and results
    EMBArk Executor->>Database: Updates analysis status
    Database-->>EMBArk Executor: Status updated
    EMBArk Web Server-->>User: Redirects to Dashboard page

Loading

Note over EMBArk Web Server: The web server also verifies your permissions as covered in Chapter 1: User Authentication & Authorization.

Key Components and Code Elements

1. embark/uploader/models.py - Defining Firmware and Analysis Blueprints: This file contains the data structures (models) for storing information about your uploaded firmware and its associated analysis.

   # Simplified snippet from embark/uploader/models.py
   from django.db import models
   import uuid
   from users.models import User as Userclass
   
   class FirmwareFile(models.Model):
       id = models.UUIDField(primary_key=True, default=uuid.uuid4)
       user = models.ForeignKey(Userclass, on_delete=models.SET_NULL, null=True)
       file = models.FileField(upload_to=...) # Stores the actual file
       # ... other fields like upload_date ...
   
   class FirmwareAnalysis(models.Model):
       id = models.UUIDField(primary_key=True, default=uuid.uuid4)
       user = models.ForeignKey(Userclass, on_delete=models.SET_NULL, null=True)
       firmware = models.ForeignKey(FirmwareFile, on_delete=models.SET_NULL, null=True)
       firmware_name = models.CharField(max_length=127, default="File unknown")
       version = models.CharField(max_length=127, blank=True)
       # ... other user-defined settings like device, architecture, emulation tests ...
       scan_modules = models.JSONField(blank=True, null=True, default=list) # Stores chosen modules
       path_to_logs = models.FilePathField(max_length=255) # Where analysis logs are stored
       status = models.JSONField(default=dict) # Tracks progress and state
       finished = models.BooleanField(default=False) # Is the analysis complete?
       # ... other metadata ...
   
       def construct_emba_command(self, image_file_location: str):
           # This method takes all the settings from this model
           # and builds the command-line arguments for the EMBA tool.
           emba_flags = self.get_flags()
           # ... (logic to determine scan profile) ...
           emba_cmd = f"cd {get_emba_root()} && {get_emba_base_cmd()} -f {image_file_location} -l {self.path_to_logs} {emba_flags}"
           return emba_cmd
   
       def get_flags(self):
           # This helper method converts model fields into EMBA command flags.
           command = ""
           if self.version:
               command += f" -X \"{self.version}\""
           if self.firmware_Architecture:
               command += f" -a {self.firmware_Architecture}"
           if self.user_emulation_test:
               command += " -E"
           if self.scan_modules:
               for module_code in self.scan_modules:
                   command += f" -m {module_code}"
           return command

   • FirmwareFile: This model represents the actual binary file uploaded by the user, storing its unique id and location.
   • FirmwareAnalysis: This is the most important model here. It holds all the specific settings (like version, firmware_Architecture, scan_modules), links to the FirmwareFile, and keeps track of metadata such as path_to_logs and the current status of the analysis.
   • construct_emba_command: This critical method collects all the settings chosen by the user in the FirmwareAnalysis object and translates them into a command-line string that the EMBA tool can directly understand and execute. Each setting (e.g., -a ARM for architecture, or -m F02 to enable a specific scan module) becomes a part of this command.

2. embark/uploader/forms.py - Building the User Interface: This file defines the FirmwareAnalysisForm, which is a Django form directly linked to the FirmwareAnalysis model.

   # Simplified snippet from embark/uploader/forms.py
   from django import forms
   from uploader import models
   
   class FirmwareAnalysisForm(forms.ModelForm):
       # Dynamically load available EMBA modules for choice fields
       MODULE_CHOICES = [
           ('f02', 'F02_toolchain'), # Example module
           ('s10', 'S10_binaries_basic_check'), # Example module
           # ... many more modules would be listed here ...
       ]
       scan_modules = forms.MultipleChoiceField(
           choices=MODULE_CHOICES,
           widget=forms.CheckboxSelectMultiple, # Allows selecting multiple options
           required=False,
           help_text='Enable/disable specific scan-modules for your analysis'
       )
   
       class Meta:
           model = models.FirmwareAnalysis
           fields = [
               'firmware', 'version', 'device', 'notes',
               'firmware_Architecture', 'user_emulation_test',
               'sbom_only_test', 'web_report', 'scan_modules'
           ]
           widgets = {
               "device": forms.CheckboxSelectMultiple, # Allows selecting multiple devices
           }

   • FirmwareAnalysisForm: This form automatically generates input fields based on the FirmwareAnalysis model's settings. It includes special fields like scan_modules that use checkboxes, making it easy for users to pick multiple EMBA modules.

3. embark/uploader/executor.py - The EMBA Launcher: This file contains the submit_firmware function, which is the main entry point for kicking off an EMBA scan.

   # Simplified snippet from embark/uploader/executor.py
   import os
   import shutil
   from uploader.models import FirmwareAnalysis, FirmwareFile
   from uploader.boundedexecutor import BoundedExecutor # Manages background tasks
   
   def submit_firmware(firmware_analysis: FirmwareAnalysis, firmware_file: FirmwareFile):
       # 1. Create a unique directory for this analysis's temporary files.
       active_analyzer_dir = f"{settings.ACTIVE_FW}/{firmware_analysis.id}/"
       # 2. Copy the uploaded firmware file into this new directory.
       shutil.copy(firmware_file.file.path, active_analyzer_dir)
   
       # 3. Create a dedicated directory for EMBA logs.
       firmware_analysis.create_log_dir() # Updates path_to_logs in the model
       firmware_analysis.set_meta_info() # Initializes the status field
   
       # 4. Build the complete EMBA command string.
       emba_cmd = firmware_analysis.construct_emba_command(image_file_location=...)
   
       # 5. Submit the EMBA command to a background executor.
       # This allows EMBArk to run the analysis without freezing the web interface.
       emba_fut = BoundedExecutor.submit(BoundedExecutor.run_emba_cmd, emba_cmd, firmware_analysis.id, active_analyzer_dir)
       # 6. Also start a log reader to monitor the analysis progress in real-time.
       BoundedExecutor.submit(LogReader, firmware_analysis.id)
   
       return bool(emba_fut) # Returns True if successfully submitted

   • submit_firmware: This function acts as the chief orchestrator. It prepares the analysis environment (creating specific folders and copying the firmware), then uses FirmwareAnalysis.construct_emba_command to build the exact command for EMBA. Finally, it hands this command over to a BoundedExecutor. This executor runs the EMBA analysis in the background, keeping the web application responsive. It also starts a LogReader to monitor EMBA's progress, which is crucial for Chapter 3: Real-time Progress Monitoring.

Conclusion

Firmware Analysis Management is the core operational engine of EMBArk. It seamlessly guides your firmware from the moment it's uploaded, through the application of precise analysis settings, and into the powerful EMBA scanning process. By managing each analysis with a unique ID and tracking all its relevant metadata, EMBArk ensures that every firmware evaluation is organized, efficient, and transparent.

Now that your firmware is uploaded and its analysis has begun, you'll want to see what's happening! Next, we'll explore how EMBArk keeps you informed about your analysis in real-time. Dive into Chapter 3: Real-time Progress Monitoring to learn how you can watch your scans unfold live!

---

_{^{Generated by AI Codebase Knowledge Builder. References: [1], [2], [3], [4], [5], [6], [7]}}