Month: December 2015

Changes in JEB 2.1… And a holiday season gift

JEB 2.1 is just around the corner! Users with a valid subscription should expect software updates today or tomorrow. This major update represents the maturation of JEB 2.0, and paves the way for JEB 2.2, which will introduce modules for x86 and ARM.

View the full change log for JEB 2.1

The following is a non-exhaustive list of notable changes.

Navigation bars in text views

The navigation bar is interactive (zoom in and out for finer granularity, visualize currently loaded text area, etc.) and customizable. It is connected to the metadata manager of an interactive unit. Client code (eg, in plugins or scripts) can manipulate this metadata: add, remove, query metadata and metadata groups, etc.

Sample script: JEB2CustomizeMetadata.py

The navigation bar can be seen on the right-hand side

Java AST API

The newly introduced com.pnfsoftware.jeb.core.units.code.java API package allows compliant Java source units to offer direct manipulation of AST code to clients. Units produced by our native Dalvik decompiler are obviously compliant. Plugins and scripts may use this API to implement complex refactoring/code-cleanup operations, as was demonstrated with JEB 1.x in the past.

The IJavaSourceUnit interface is your entry-point to AST elements

Note that the Java AST API has changed significantly relative to JEB1’s. A few missing features will be implemented in future service releases of JEB 2.1.x (eg, tagging) but overall, it is more powerful than JEB1’s: most objects can be constructed and modified, AST elements that were not offered by the older API, such as Annotations, are now accessible, etc. – not to mention, those units are now persisted! (See our next section.)

Sample scripts:

Improved persistence

Semi-related to the above paragraph, we are glad to announce that the decompiled Java code (and all the modifications applied to it via the Java AST API) are now persisted to the JDB2 file when saving a project to file.

The persistence mechanism has undergone significant changes and fixes, and therefore, some JDB2 generated by JEB 2.0 might not be compatible with JEB 2.1.

The newer version of the PDF plugin also supports persistence.

API for UI scripting

The IGraphicalClientContext has been augmented to support more operations, such as enumerating views and fragments, setting the focus on a specific view, setting or retrieving the active address and active items, etc.

Sample scripts:

A sample UI script showing how to start an asynchronous, interruptible task

The official RCP client implements the UI-API. We are planning to add more UI primitives in the upcoming maintenance releases.

API changes

Here is an incomplete list of API changes that took place between the latest 2.0 and the initial 2.1 releases:

  • Most protected members of the AbstractUnit hierarchy of skeleton classes have been privatized.  The current guidelines is to use the provided getters, setters, and other accessor methods.
    • Side-note: As always, we encourage plugin developers to use abstract implementations instead of implementing interfaces from scratch, when abstracts are available.
  • Units offer a way to persist or not-persist their children units when saving a project to disk. Changes took place in IUnit.addChild() and co. methods.
  • We added a notification manager (for all units) and introduced a metadata manager (for interactive units only).
  • The formatter (aka, the unit output manager) was also revamped: it can now yield transient and persisted documents. Write-access is also permitted, which means that plugins and scripts can add documents (such as texts, tables, or trees), and request that they’re persisted in the JDB2 database. We uploaded sample scripts here:
  • The client notifications are now called ClientNotification, to avoid potential confusion with another type of notification used within JEB (the unit notifications).

Next up

We are planning a few maintenance updates before the release of JEB 2.2. The currently planned release date for JEB 2.2 is early February 2016.

We will keep you posted on this very blog. Stay tuned, and a happy holiday season to all.

PS: as an early Christmas gift, we have uploaded a new third-party plugin on our public repository. Check out the Antivirus Quarantines File Extractor plugin. We currently support Kaspersky KLQ quarantine files only, but are planning to add more soon. If you’d like to contribute, please send us an email.

Scanning PDF Files using JEB2

Update (9/13/2017): we open-sourced the PDF plugin. A compiled JAR binary is also available.

Update: Feb. 27: Slides – Automation How-To
Update: Dec. 3: List of notifications

In this blog post, we show how JEB2 can be used as a building block of a file analysis system. We will show how to use the Core API to create a headless client. That client will scan PDF files using the JEB2 PDF Analysis Module. Basics of the IUnit and co. interfaces is also demonstrated.

Source code on GitHub.

Sample execution output produced by the PDF Scanner

As this slide deck shows, the back-end and front-end components of JEB2 are separated. The official RCP desktop client uses the JEB2 Core API; other front-ends, like the PDF scanner, can be built using that same API.

JEB2 HL Architecture Diagram

Creating an Eclipse project

Let’s get started by creating a new code project. We will show how to do this in Eclipse.

0- Check your license of JEB2. Make sure to use a license that supports third-party client creation and the loading of third-party plugin. If you haven’t done so, download and drop the PDF module in your coreplugins/ sub-directory.

1- Clone our sample code repository: git clone https://github.com/pnfsoftware/jeb2-samplecode.git

2- Create a new Java project. The Java source folder should be rooted in the src/ directory.

3- Add the JEB2 back-end as a JAR dependency. The back-end software is contained in the file bin/cl/jeb.jar located within your installation folder. You may also want to link that JAR to the API documentation, contained in the doc/apidoc.jar file, or online at https://www.pnfsoftware.com/jeb/apidoc

Your Package Explorer view should now look like:

Package explorer view after setting up dependencies

5- Set up the execution options. The required Java properties for execution (jeb.engcfg and jeb.lickey) can be set in the Run Configurations panel (accessible via the Run menu). Example:

Example of a Run configuration

6- Open the com.pnf.pdfscan.PDFScanner source file. You are ready to execute main().

How the scanner works

Now, let’s focus on the scanner source code.

  • The JEB2 back-end is initialized when scanFiles() is called:
    • Use JebCoreService to retrieve an instance to ICoreContext
    • Create an IEnginesContext
    • Load a project within that context (IRuntimeProject)
    • Add artifact(s) and process them (ILiveArtifact)
      • We add a single file artifact per project in this example
    • Retrieve the products (IUnit)
      • We are retrieving the top-most unit only in this example
    • Analyze the unit (see assessPdf())
    • Close the project

[Note: A detailed explanation of the above concepts (core, engines, project, artifacts, units, etc.) is outside the scope of this tutorial. Refer to our Developer Portal for more information.]

Snippet of scanFiles()

The assessPdf() method evaluates PDF units. The evaluation performed by this sample scanner is trivial: we collect the notifications created by the PDF plugin during the analysis of the file, and see if they meet basic criteria.

About the Unit Notifications:

  • Any JEB2 plugin can attach notifications to its units. The PDF plugin does so. Notifications are meant to pin-point noteworthy areas of a unit or artifiact.
  • A notification has a “dangerosity level” ranging from 0 to 100. It also has a description, an optional address to point to which area of the unit the notification is associated with, etc.
  • The API offers standard notification types, ranging from “Interesting area” to “Definitely Malicious”.
Standard notification levels offered in the NotificationType enum

A PDF unit can contain several types of notifications. Example include: corrupt areas in stream; multiple encoding of stream; JavaScript; password-protected stream; invalid/illegal entries in stream; etc.

Link: Complete list of notifications issued by the PDF plugin.

Our simple scanner reports a file as suspicious if it contains at least 2 notifications that have a level >= 70 (POTENTIALLY_HARMFUL). These thresholds can be tweaked in the source code.

The assessPdf() routine

The screenshot below is a sample output produced by the PDF scanner:

Conclusion

The intent of this entry is to shed some light on the process of writing third-party clients for JEB2, as well as what and how to use notifications reported by Units. We encourage you to visit our Developer Portal to find additional documentations as well as the reference Javadoc of the API.