User interface how-to in JEB

The release of JEB 2.1.2 is being distributed to our customers today and tomorrow. We thought it would be a good time to present/recap some of the UI changes that were introduced since version 2.1.

Layouts

The RCP client comes with a default layout that has the Project view on the left-hand side, the Logger and Console at the bottom, and a large empty workspace area in the center. The layout can (and should!) be customized to fit your analysis needs.

Drag views around by their title areas. Expand a view to full-screen by double clicking on its title area. Minimize or maximize view groups using the icons located in the view trimbar. (Circled in red in the picture below.)

Customized layout with a code hierarchy on the lower-left corner.

Since you may want to have different layouts for different use cases, layouts can be duplicated and customized. You can achieve this via the Window/New Layout… menu option.

Auto-sync the Project tree selection

Enable this feature via the double-arrow icon located in the Project Explorer view. (As seen on the picture below.) When enabled, the simple selection of a unit element in the tree will automatically bring up the associated unit view. No need for Enter, no need for double-click: a simple selection is enough.

This option is especially useful when navigating large swarm of resource files, eg pictures.

Open same-type unit in same views

When enabled, a unit of view X will be opened in an already existing view representing another unit of the same type (X).

This option is extremely useful when opening many views of the same type, but only the last one is important: example, when decompiling and navigating code.

Navigating a text view with a non-sticky caret

By default, the navigation of a text view in JEB2 may be a bit confusing: due to the way very large buffers are handled by these views, it is often more resource-efficient to keep the caret on its viewport location. That means that, upon scrolling up or down, the caret will visually remain where it is.

When highlighting interactive items, and wanting to keep track of other related items across the buffer, that default behavior is not ideal: it is better to maintain the caret position within the buffer, as opposed to within the viewport.

Use Control (Control on Mac) +Shift + Up|Down to keep the caret where it is when scrolling up/down.

More to come

We will keep this entry updated as we add more how-to and gotchas regarding the RCP client user interface. If you have questions or requests, feel free to email us at support@pnfsoftware.com.

Version 0.2.9 of the PDF analyzer plugin is available

Update (9/13/2017): we open-sourced the PDF plugin. A compiled JAR binary is also available.

We have released version 0.2.9 of our PDF analyzer plugin for JEB2. This release adds support for XFA (XML Forms Architecture) fragment streams reconstruction and parsing.

In the following example, a malicious PDF file contains two XFA streams encoded with the unusual CCITTFFax encoder. Once decoded, JEB2 reassembles the decoded contents into a unit “32 0”. The XFA contains a malicious JavaScript snippet, also visible as a separate unit:

Reconstructed XFA data showing a malicious JavaScript snippet.

Notifications reported also show a dangerous Open action.

The malicious PDF file examined in this entry is available on VirusTotal.
SHA256: e108432dd9dad6ff57c8de6e907fd6dd25b62673bd4799fa1a47b200db5acf7c

Changes in JEB 2.1… And a holiday season gift

JEB 2.1 is just around the corner! Users with a valid subscription should expect software updates today or tomorrow. This major update represents the maturation of JEB 2.0, and paves the way for JEB 2.2, which will introduce modules for x86 and ARM.

View the full change log for JEB 2.1

The following is a non-exhaustive list of notable changes.

Navigation bars in text views

The navigation bar is interactive (zoom in and out for finer granularity, visualize currently loaded text area, etc.) and customizable. It is connected to the metadata manager of an interactive unit. Client code (eg, in plugins or scripts) can manipulate this metadata: add, remove, query metadata and metadata groups, etc.

Sample script: JEB2CustomizeMetadata.py

The navigation bar can be seen on the right-hand side

Java AST API

The newly introduced com.pnfsoftware.jeb.core.units.code.java API package allows compliant Java source units to offer direct manipulation of AST code to clients. Units produced by our native Dalvik decompiler are obviously compliant. Plugins and scripts may use this API to implement complex refactoring/code-cleanup operations, as was demonstrated with JEB 1.x in the past.

The IJavaSourceUnit interface is your entry-point to AST elements

Note that the Java AST API has changed significantly relative to JEB1’s. A few missing features will be implemented in future service releases of JEB 2.1.x (eg, tagging) but overall, it is more powerful than JEB1’s: most objects can be constructed and modified, AST elements that were not offered by the older API, such as Annotations, are now accessible, etc. – not to mention, those units are now persisted! (See our next section.)

Sample scripts:

Improved persistence

Semi-related to the above paragraph, we are glad to announce that the decompiled Java code (and all the modifications applied to it via the Java AST API) are now persisted to the JDB2 file when saving a project to file.

The persistence mechanism has undergone significant changes and fixes, and therefore, some JDB2 generated by JEB 2.0 might not be compatible with JEB 2.1.

The newer version of the PDF plugin also supports persistence.

API for UI scripting

The IGraphicalClientContext has been augmented to support more operations, such as enumerating views and fragments, setting the focus on a specific view, setting or retrieving the active address and active items, etc.

Sample scripts:

A sample UI script showing how to start an asynchronous, interruptible task

The official RCP client implements the UI-API. We are planning to add more UI primitives in the upcoming maintenance releases.

API changes

Here is an incomplete list of API changes that took place between the latest 2.0 and the initial 2.1 releases:

Most protected members of the AbstractUnit hierarchy of skeleton classes have been privatized. The current guidelines is to use the provided getters, setters, and other accessor methods.
- Side-note: As always, we encourage plugin developers to use abstract implementations instead of implementing interfaces from scratch, when abstracts are available.
Units offer a way to persist or not-persist their children units when saving a project to disk. Changes took place in IUnit.addChild() and co. methods.
We added a notification manager (for all units) and introduced a metadata manager (for interactive units only).
The formatter (aka, the unit output manager) was also revamped: it can now yield transient and persisted documents. Write-access is also permitted, which means that plugins and scripts can add documents (such as texts, tables, or trees), and request that they’re persisted in the JDB2 database. We uploaded sample scripts here:
- JEB2ExtraDocument.py
- JEB2ExtraDocumentTableTree.py
The client notifications are now called ClientNotification, to avoid potential confusion with another type of notification used within JEB (the unit notifications).

Next up

We are planning a few maintenance updates before the release of JEB 2.2. The currently planned release date for JEB 2.2 is early February 2016.

We will keep you posted on this very blog. Stay tuned, and a happy holiday season to all.

PS: as an early Christmas gift, we have uploaded a new third-party plugin on our public repository. Check out the Antivirus Quarantines File Extractor plugin. We currently support Kaspersky KLQ quarantine files only, but are planning to add more soon. If you’d like to contribute, please send us an email.

Scanning PDF Files using JEB2

Update (9/13/2017): we open-sourced the PDF plugin. A compiled JAR binary is also available.

Update: Feb. 27: Slides – Automation How-To
Update: Dec. 3: List of notifications

In this blog post, we show how JEB2 can be used as a building block of a file analysis system. We will show how to use the Core API to create a headless client. That client will scan PDF files using the JEB2 PDF Analysis Module. Basics of the IUnit and co. interfaces is also demonstrated.

Source code on GitHub.

Sample execution output produced by the PDF Scanner

As this slide deck shows, the back-end and front-end components of JEB2 are separated. The official RCP desktop client uses the JEB2 Core API; other front-ends, like the PDF scanner, can be built using that same API.

Creating an Eclipse project

Let’s get started by creating a new code project. We will show how to do this in Eclipse.

0- Check your license of JEB2. Make sure to use a license that supports third-party client creation and the loading of third-party plugin. If you haven’t done so, download and drop the PDF module in your coreplugins/ sub-directory.

1- Clone our sample code repository: git clone https://github.com/pnfsoftware/jeb2-samplecode.git

2- Create a new Java project. The Java source folder should be rooted in the src/ directory.

3- Add the JEB2 back-end as a JAR dependency. The back-end software is contained in the file bin/cl/jeb.jar located within your installation folder. You may also want to link that JAR to the API documentation, contained in the doc/apidoc.jar file, or online at https://www.pnfsoftware.com/jeb/apidoc

Your Package Explorer view should now look like:

Package explorer view after setting up dependencies

5- Set up the execution options. The required Java properties for execution (jeb.engcfg and jeb.lickey) can be set in the Run Configurations panel (accessible via the Run menu). Example:

6- Open the com.pnf.pdfscan.PDFScanner source file. You are ready to execute main().

How the scanner works

Now, let’s focus on the scanner source code.

The JEB2 back-end is initialized when scanFiles() is called:
- Use JebCoreService to retrieve an instance to ICoreContext
- Create an IEnginesContext
- Load a project within that context (IRuntimeProject)
- Add artifact(s) and process them (ILiveArtifact)
  - We add a single file artifact per project in this example
- Retrieve the products (IUnit)
  - We are retrieving the top-most unit only in this example
- Analyze the unit (see assessPdf())
- Close the project

[Note: A detailed explanation of the above concepts (core, engines, project, artifacts, units, etc.) is outside the scope of this tutorial. Refer to our Developer Portal for more information.]

The assessPdf() method evaluates PDF units. The evaluation performed by this sample scanner is trivial: we collect the notifications created by the PDF plugin during the analysis of the file, and see if they meet basic criteria.

About the Unit Notifications:

Any JEB2 plugin can attach notifications to its units. The PDF plugin does so. Notifications are meant to pin-point noteworthy areas of a unit or artifiact.
A notification has a “dangerosity level” ranging from 0 to 100. It also has a description, an optional address to point to which area of the unit the notification is associated with, etc.
The API offers standard notification types, ranging from “Interesting area” to “Definitely Malicious”.

Standard notification levels offered in the NotificationType enum

A PDF unit can contain several types of notifications. Example include: corrupt areas in stream; multiple encoding of stream; JavaScript; password-protected stream; invalid/illegal entries in stream; etc.

Link: Complete list of notifications issued by the PDF plugin.

Our simple scanner reports a file as suspicious if it contains at least 2 notifications that have a level >= 70 (POTENTIALLY_HARMFUL). These thresholds can be tweaked in the source code.

The screenshot below is a sample output produced by the PDF scanner:

Conclusion

The intent of this entry is to shed some light on the process of writing third-party clients for JEB2, as well as what and how to use notifications reported by Units. We encourage you to visit our Developer Portal to find additional documentations as well as the reference Javadoc of the API.

Writing client scripts for JEB2 using Python

The latest release of JEB2, version 2.0.14, introduces a feature familiar to JEB1 users: client scripts written in Python.

Both Standard and Business licenses permit running scripts. They can be written using the Python 2.5 or 2.7 syntax and features, and are executed by Jython. (A Jython stand-alone package is required to run scripts. We recommend version 2.5. Download it and drop it in your JEB2 scripts/ sub-directory.)

Feature-wise, scripts use the standard JEB2 core APIs. They are also using the client API, available in the com.pnfsoftware.jeb.client.api package. As usual, refer to our Developer Portal and Javadoc website for API reference and usage.

A client script implements the IScript interface. Upon execution, the script run() entry-point method is provided an IClientContext or derived object, such as an IGraphicalClientContext for UI clients. (The official RCP desktop client falls in the latter category.)

Here is the simplest of all scripts:

from com.pnfsoftware.jeb.client.api import IScript
class JEB2SampleScript(IScript):
  def run(self, ctx):
    print('Hello, JEB2')

Within the official desktop client, scripts can be executed via the File, Scripts menu item.

Finally, remember that scripts are meant to execute small, light-weight actions. Heavy lifting operations (such as parsing or background event-driven tasks) should be implemented by back-end plugins in Java.

Check out our GitHub repository for more sample scripts.

Developing JEB2 parsers and plugins

Update (11/2): parts 7 and 8 are available.

Our tutorials are available on the JEB2 developer portal, which aggregates all resources for API developers:

JEB2 Developer Portal

Setting up JEB2 to parse optimized DEX (odex) files

This blog assumes that JEB version 2.1.0 or above is used along with the OAT plugin 1.0.2 or above.

Parsing support for optimized DEX files was added to JEB2 to allow the analysis of non-deodex’ed files. Since ODEX files are target-dependant, the executing Dalvik VM is no longer restricted to regular opcodes. ODEX files may make use of “illegal” opcodes, optimized opcodes, or even the once regular but now dead extended opcodes. Whenever possible, parsing will take place, and instructions displayed in the assembly view.

In the screenshot below, note that opcode 43h (illegal for non-optimized code) is used, as well as iput-wide-volatile (optimized opcode for field access).

In that second screenshot, notice the use of a non-standard jumbo opcode.

If you are analyzing an extracted ODEX file (one whose header bytes start with “dey\n”), then all versions of JEB2 shall be able to process it. The Project tree will look like the following (project > artifact > odex unit > dex unit):

If you are analyzing an OAT file (DEX file precompiled to native and ready to run within the ART runtime), then you will need one additional plugin: the OAT plugin. This plugin can be registered on Business and Enterprise versions of JEB2. (Note: older versions of JEB 2.0, mainly versions 2.0.12 and above, require the third-party ELF plugin as well.)

Installation steps:

Visit our public GitHub account
Download the latest package of the OAT plugin
Drop the JAR file in the coreplugins folder within your JEB2 installation directory
Restart JEB2. The lines “Plugin loaded … OATPlugin” should be visible in the console

The OAT plugin is loaded (the ELF third-party plugin is no longer required with JEB 2.1+)

Now, you may open an OAT file. The project view should be similar to the following (project > artifact > elf unit > oat unit > dex or odex unit):

Here is another example of an ELF file containing an OAT section, containing 2 optimized DEX files:

That is it for this blog post. We are planning to release more documentation and tutorials about our APIs In the coming days. In the meantime, remember to check our open-source plugins on GitHub, they are great starting points for anyone interested in writing their own parsers or back-end plugins. Stay tuned, and happy analysis.

JEB2 introduction videos

With the constant improvement of our UI client, new plugins, the upcoming release of the Dalvik debugger, as well as the APIs, JEB2 has become increasingly complex and full-featured.

In order to reduce the barrier of entry for new users, we are going to publish a series of introductory videos aiming to explain how to get started with JEB2 and how to use the RCP client to the fullest. We will strive to keep the videos short and concise, highlighting specific features and gotchas of our software.

You will find the YouTube “JEB2 Discovery” playlist here:

JEB2 available to all customers

Dear users,

We are glad to announce that JEB2 is now available to all our customers!

If you have a valid JEB 1.x license, you should have received an equivalent build with a subscription valid till the end of your support period. If this is not the case, please reach out to support@pnfsoftware.com.

Additional documentation and resources can be found on our website. We are also providing PDF analysis support to all business/enterprise customers! The JEB2 PDF plugin can be downloaded here: https://www.pnfsoftware.com/jeb/plugins.

Open-source plugins and PoC plugins such as Linux ELF, Android OAT, MIPS, Microsoft OLE, XLS, PPT, etc. can and will be found on our GitHub repository here: https://github.com/pnfsoftware.

The back-end APIs will be made available in the Fall 2015.

Note: This message was originally published on our Google Groups forum.

Android Dalvik, inside OAT, inside ELF

As a follow up to our last blog on Adobe PDF and Microsoft XLS plugins for JEB2, here is another example of deep analysis support in the case of nested artifacts, as is the case for Android pre-compiled native apps.

Those apps are run using ART, the newest Android Runtime. They are native Linux ELF .so files, embedding a custom Android OAT file, which in turns contains one or more Dalvik DEX bytecode files. See the pictures below for an example:

The ELF and OAT plugins will be open sourced.

Stay tuned for more news within the next few days!