Debugging Android apps on Android Pie and above

Update (May 2024): Debugging oddities with API levels 34+, native code debugging issues with API level 34. Refer to the end of this post.

Update (March 2020): The original post describing the impossibility to read locals that do not have associated DebugLocalInfo on Android 9 (P) and 10 (Q) was fixed in Android 11 (R).

[Original Post] Issues related to reading local vars

Lower-level components of the Dalvik debugging stack, namely JDWP, JVM TI, and JVM DI implementations, were upgraded in Android Pie. It is something we indirectly noticed after installing P.beta-1 in the Spring of 2018. For lack of time, and because our recommendation is to debug apps (non-debuggable and debuggable alike) using API levels 21 (Lollipop) to 27 (Oreo), reversers could easily avoid road blocks which manifested in JEB as the following:

An empty local variable panel (with the exception of this for non-static methods)
Type 35 JDWP errors reported in the console, indicating that an invalid slot was being accessed

A type 35 error in this context means an invalid local slot is being accessed. In the example shown above, it would mean accessing a slot outside of [0, 10] (per the .registers directive) since the method declares a frame of 11 registers.

The second type of noticeable errors (not visible in the screenshot) were mix-ups between variable indices. Normally, and up to the JDWP implementation used in Android Pie, indices used to access slots were Java-style parameter indices (represented in Dalvik as pX), instead of Dalvik-style indices (vX). Converting from one to the other is trivial assuming the method staticity and prototype is known. It is a matter of generating pX so that they end up at the bottom of the frame. In the case above:

v0    p2
v1    p3
v2    p4
..
v9
v10   p0
v11   p1

When issuing a JDWP request (16,1) to read frame slots, we would normally use pX indices. It is no longer the case with Android P and Q: vX indices are to be used.

Open up JEB, start debugging any APK, switch to the Terminal view, and type ‘info’. In the context of JDWP, this JEB Debugger command issues Info and Sizes requests. Notice the differences:

=> On Android Oreo (API 27):

VM> info
Debuggee is running on ?
VM information: JDWP:"Android Runtime 2.1.0" v1.6 (VM:Dalvik v1.6.0)
VM identifier sizes: f=8,m=8,o=8,rt=8,fr=8

=> On Android Pie (API 28) and Android Q:

VM> info
Debuggee is running on ?
VM information: JDWP:"Java Debug Wire Protocol (Reference Implementation) version 1.8
JVM Debug Interface version 1.2
JVM version 0 (Dalvik, )" v1.8 (VM:Dalvik v0)
VM identifier sizes: f=4,m=4,o=8,rt=8,fr=8

Notice the reported version 1.2 for JVM DI, previously unspecified, and reported version 1.8 for JDWP, likely the cause of the breakage. Also note ID encoding size updates. JDWP had been reported a 1.6 version number, as well as field and method IDs encoded on 8 bytes, for as long as I can remember.

The vX/pX index issue was easily solved. It took a little while to crack the second issue. A superficial browsing of AOSP did not show anything fruitful, but after digging around, it seemed clear that this updated implementation of JDWP used CodeItem variables’ debug information to determine which variables are worth checking, and using what type.

In JEB, right-click and select Rendering Options, tick Show Debug Directives to display variable definition and re-definition information. In the example above, the APK holds information stating that v0 is being using as a boolean starting at address 2, and v1 a String starting at address 4. Android P+’s JDWP implementation does use this information to validate local variables accesses.

See below: at address 2, v0 has been declared and is rendered. v1 has not been declared yet, the debugger cannot read it (we’ll get error 35).

Single-step: at address 4, v1 is declared. Although it is uninitialized, the debugger can successfully read the var:

So – Up until P, this metadata information, when present (almost all Release-type builds of legitimate and malware files alike discard it), had been considered indicative. Now, the debugger takes it literally. There are multiple candidate reasons as of why, but an obvious one is Safety. JDWP has been known to have the potential to crash the VM when receiving reading requests for frame variables using a bad type. E.g., requesting to read an integer-holding slot as a reference would most likely crash the target VM. Using type information providing in metadata, a debugger server can now ensure that a debugger requesting to read a slot as type T is indeed a valid request – assuming the metadata is legitimate, and since the primary use case is to debug applications inside IDE, which hold source information used to generate valid debug metadata, the assumption is fair.

Validating access to local vars has the interesting side-effect to act as an anti-debugging feature. While debugging the app remains possible, not being able to easily read some locals (parameters pX are always readable though), can be quite an annoyance.

In the future, how could we work around the JDWP limitation? Well, aside from the obvious cop out “use Oreo or below”, an idea would be to extend JEB’s –makeapkdebug option (that generates a debuggable version of a non-debuggable APK) to insert DEX metadata information specifying that all variables of a frame are used and of a given type. That may not work depending on the type of validation performed by the DEX verifier, but it’s something worth exploring. Maybe more simply, an alternative could be a custom AOSP build that disabled that feature. Or better yet, finding if a system property exists to disable/enable that JDWP functionality.

A final note: debugging non-debuggable APKs on Android Pie or above also proved more difficult, if not practically impossible, than on Oreo and below. Assuming your phone is rooted, here’s a solution (found when browsing around AOSP commits). On a rooted phone:

> adb root
> adb shell setprop dalvik.vm.dex2oat-flags --debuggable
> adb shell stop
> adb shell start

[May 2024] JDWP DDMS packet on Android 14 and above

With API levels 34+ (Android 14 and above), JEB 5.12 and below may report JDWP errors regarding unhandled JDWP packets:

[E] Unknown JDWP command packet: [JDWP:id=19,fl=00h,cc=(-57,1),dl=14016]

The command set -57 is a custom JDWP command set for Android, used for DDMS (Dalvik Debug Monitor System). JEB does not support those requests. Starting with JEB 5.13, those target-issued requests will be made more explicit (specifically saying they are DDM messages), but also less verbose (below INFO level) to avoid cluttering the logger.

[May 2024] Problems debugging native code on Android 14

JEB has difficulties debugging native code with Android 14 / API level 34. The native threads may suspend over an invalid memory access (SIGSEGV, signal 11) received when simply interacting with the app:

[I] [GDB] [SIGNAL] type=GENERIC signal=11 tid=7185 @ 58C00ECCh

The issue is only present with API 34. It is not present with API levels <=33 and seems to have disappeared with the current preview of API 35 (upcoming Vanilla Ice Cream release).

We do not have a workaround in place at this time, and can only recommend avoiding debugging of native code with Android 14 / API level 34. We will update this post if/when a workaround is found.

Dynamic JNI Detection Plugin

Update (Nov 29): the plugin was open-sourced on our GitHub repository. JEB 3.0.7+ is required to load and run it.

Java applications can call native methods stored in dynamic libraries via the Java Native Interface (JNI) framework. Android apps can do the same: developers can use the NDK to write their own .so library to use and distribute.

In this post, we briefly present how the binding mechanisms work, allowing a piece of bytecode to invoke native code routines.

Named Convention Method

The easiest way to call native method is as such:

In Java, class com.example.hellojni.HelloJni:

In C:

The native method name adheres to the standard JNI naming convention, allowing automatic resolution and binding.

The corresponding Dalvik bytecode is:

and here are the the corresponding ARM instructions:

JEB automatically binds those methods together, to allow easy debugging from bytecode to native code.

However, there is another way to bind native code to Java.

Dynamic JNI Method

One can decide to bind any function to Java without adhering to the naming convention, by using the JNIEnv->RegisterNatives method.

For example, the following line of code dynamically binds the Java method add(II)I to the native method add():

Due to its dynamic nature, statically resolving those bindings can prove difficult in practice, e.g. if names were removed or mangled, or if the code is obfuscated. Therefore, not all calls to RegisterNatives may be found and/or successfully processed.

However, JEB 3.0-beta.2 (to be released this week) ships with an EnginesPlugin to heuristically detect – some of – these methods, and perform binding – and of course, you will also be able to debug into them.

Execute the plugin via the File, Plugins menu

Once run, it will :

annotate the dex code with the target addresses:

rename targets (prefixing names with __jni_) :

enable you to seamlessly debug into them (jump from Java to this JNI method)

Heuristics

As of this writing, the plugin uses several heuristics, implemented for ARM and ARM64 (Aarch64):

The first is the simplest one: the JNIEnv->RegisterNatives method is commonly called from the standard JNI initialization function JNI_OnLoad, so JEB searches for this method and attempt to find calls to RegisterNatives.

Once the ‘BL RegisterNatives‘ is found, JEB uses the decompiler to create an IR representation of the block, and determines the values of R2 and R3 (X2 and X3 on Aarch64). R3 indicates the number of native methods to register, R2 is a pointer to the array of JNI native methods (structure with a pointer to method name, a pointer to method signature and a pointer to the native function bound):

Even if accurate, this method does not work when a Branch is issued via a register (BL R4) or method name is hidden.

The second heuristic is based on method name. First, in Dalvik, we search for all invocations to native methods. Then, for each method found, we search in binaries if there is a String reference matching the method name. (This heuristic is dangerous but yields decent results. A future plugin update may allow users to disable it.)

If found, the plugin looks at cross references of this String and checks if it looks like the expected JNI structure.

The third and last heuristic is the same as the previous one, but based on arguments. Since names can be shortened, they may not be interpreted as String, and thus not referenced, whereas it is easier to find argument signatures.

These three heuristics only work when methods are defined as a static array variable. Dynamic variables would need some emulation of the JNI_OnLoad method to be resolved.

As you can see, detection is currently based on heuristics, so obfuscated methods may be missing. Feel free to tweak and improve the plugin, it is available on our GitHub repository. As usual, feel free to reach out to us (email, Twitter, Slack) if you have questions or suggestions.

Debugging Dynamically Loaded DEX Bytecode Files

The JEB 2.3.2 release contains several enhancements of our JDWP and GDB/LLDB¹ debugger clients used to debug both the Dalvik bytecode and native code of Android applications.

Dynamically loaded DEX files

In this post, we wanted to highlight a neat addition to our Dalvik debugger. Up until now, we did not support debugging several DEX files within a single debugging session. ²

So, we decided to add support for debugging DEX files loaded in a dynamic fashion. Below is a use-case, step-by-step study of a simple app whose workflow goes along these lines:

A routine in the principal classes.dex file looks for an encrypted asset
That asset is extracted and decrypted; it is a Jar file containing additional DEX bytecode
The Jar file is dynamically loaded using DexClassLoader, and its code is executed

Now, we want to debug that additional bytecode. How do we proceed?

An example of debugging dynamically loaded bytecode

The app is called EnDyna (a benign crackme-like app, download it here). It offers a simple text box, and waits for the user to input a passcode. When entering the proper passcode, a success message is displayed.

Open the app in JEB. It contains a seemingly-encrypted asset file called edd.bin.

A closer look at the MainActivity class shows that the edd.bin file is extracted, decrypted (using a simple XOR cipher) and loaded using DexClassLoader in order to validate the user input.

Let’s attach the debugger to the app, and set a breakpoint where the call to the DexClassLoader constuctor is made.

A breakpoint was set on the DexClassLoader constructor invocation

We then trigger the verify() routine by inputting a passcode and hitting the Verify button. Our breakpoint is immediately hit. By examining the stackframe of the paused thread, we can retrieve the class loader variables and see where the decrypted DEX file was written to – and is about to get loaded from.

The decrypted Jar file about to be loaded from the path referenced by the stack variable v8

We use the Dalvik debugger interpreter to retrieve the file (command “pull”).

We now have the Jar file containing our dynamically-loaded DEX file in hand! We load it in JEB by adding an additional artifact to the project (command File, Add an Artifact…).

After processing is complete, the Android debugger notices that the added artifact contains a DEX file, and integrates it in its list of managed units.

We can set a breakpoint on the method of the second DEX file that’s about to be called.³

The second DEX file; notice the decompiled chk() method on the right-side. Here, we set a breakpoint on the method’s first instruction. It’s about to be called from MainActivity.verify(), in the primary classes.dex file.

We resume execution, our breakpoint is hit: we can start debugging the dynamically dropped DEX file!

Of course, all of the above actions can be automated by a Python script or a Java plugin. (We will upload a sample script that hooks DexClassLoader on our public GitHub repository shortly.)

We published a short video that demos the above steps, have a look at it if you want to know precisely the steps that we took to get to debug the additional DEX file.

Thank you – stay tuned for more updates, and happy debugging!

Our native GDB debugger client underwent a major revamp, as we upgraded to the LLDB debugger server instead of gdbserver. More details in a separate post! ↩
It was a non-issue for standard multi-DEX APKs since JEB automatically merges them into a single, virtual DEX file, bypassing the 64Kref limits if it has to ↩
Note that the class in question (com.xyz.kf.Ver) may not even be loaded at this point; it is perfectly fine to do so: JEB handles dynamically loaded types fine and will register breakpoints timely and accordingly. ↩

Advanced Debugger Usage via the Interpreter

This blog is a follow-up to the original debuggers tutorial published here. It focuses on the debuggers’ command interpreters, and how to use the new advanced features introduced in JEB 2.2.10 to make the most out of your live debugging sessions.

Context

When debugging with JEB, you have the standard basic visual options to set/unset breakpoints, step and 3 tabs that gives some hints on the device being debugged

So let’s see today a feature that has been newly refreshed: the interpreter. It is available in the Console window

The first command to learn is help: it lists all the available commands. The Console part, as well as the Logger part, is common to all debuggers. So the first step is to select the debugger to work with.

Type list to see available debuggers

In this case, we have 2 active debuggers: VM (for java code) and Process (for native code). Let’s connect to the VM debugger (it is always available when you debug an android project). Type use 1. The prompt changes to indicate the current debugger. Use exit to go back to root level.

There are 2 types of commands in the VM interpreter: thread related commands and object interaction commands.

VM Commands

Here is a summary of the thread management commands. It consists of a set of debugger basic features that are available in the UI: resume, step, manage breakpoints.

By pressing enter without any command, you can repeat the same action, which is pretty handy for step actions.

But that’s not all, the interpreter has some more powerful commands to manage objects/classes and interact with them. Let’s take a look at these commands

To get more help on a command, type “help $command$”, there will be a full description of it.

Class Commands

Now, here is the interesting step: you can retrieve fields/methods from a cid. A cid (stands for classId) represents a way to access the class. It can be:

the signature of this class (Ljava/lang/String;),
the inner class id prefixed with ‘c’ character

VM> methods Ljava/lang/String;

But it can also be inferred from an object:

most common this object
the object id prefixed with ‘@’ character
any object that can be referenced from these ones.

VM> methods this.mActionBar

JEB interpreter supports auto completion using TAB key.

By default, the interpreter only retrieves fields and methods from the current class (not all from superclasses and interfaces). There are 2 options for this:

-r will recursively process the parent superclasses
-i will display interfaces of the class (it can also be combined with -r to retrieve all interfaces of the superclasses)

It is also possible to select the Thread and the frame you want to work with (is case you use the this object):

-t threadId to select the thread. Remember that you can see all threads with command threads.
-f frameIndex to select the frame. By default, the top frame is used.

Object Commands

The read command provides the same features as the VM/Local tree using the command line.

All commands implement the -b option that prints out a brief output in one or two lines.

The set command allow set native types, string and arrays (object creation is not supported for now)

VM> set this.vArrayInt [I{1, 2, 3}

VM> set this.mystring "new content"

Finally, the call command allow invoking any method on objects or classes. Arguments must be separated by commas.

Conclusion

Visit the JEB public Google groups if you have additional questions on how to use the debuggers’ interpreters. Thank you for your support.

Defeating an Android application protector

AppSolid is a cloud-based service designed to protect Android apps against reverse-engineering. According to the editor’s website, the app protector is both a vulnerability scanner as well as a protector and metrics tracker.

This blog shows how to retrieve the original bytecode of a protected application. Grab the latest version of JEB (2.2.5, released today) if you’d like to try this yourself.

Bytecode Component

Once protected, the Android app is wrapped around a custom DEX and set of SO native files. The manifest is transformed as follows:

Manifest of a protected app. The red boxes indicate additions. The red line indicates removal of the LAUNCHER category from the original starting activity

The package name remains unchanged
The application entry is augmented with a name attribute; the name attribute references an android.app.Application class that is called when the app is initialized (that is, before activities’ onCreate)
The activity list also remain the same, with the exception of the MAIN category-filtered activity (the one triggered when a user opens the app from a launcher)
A couple of app protector-specific activity are added, mainly the com.seworks.medusah.MainActivity, filtered as the MAIN one

Note that the app is not debuggable, but JEB handles that just fine on ARM architectures (both for the bytecode and the native code components). You will need a rooted phone though.

The app structure itself changes quite a bit. Most notably, the original DEX code is gone.

Structure of the protected app. The fake PNG file contains encrypted assets of the original app, which are handled by libmd.so.

An native library was inserted and is responsible for retrieving and extracting the original DEX file. It also performs various anti-debugging tricks designed to thwart debuggers (JEB is equipped to deal with those)
A fake PNG image file contains an encrypted copy of the original DEX file; that file will be pulled out and swapped in the app process during the unwrapping process

Snippet of high_rezolution.png – 0xDEADCODE

Upon starting the protected app, a com.seworks.medusah.app object is instantiated. The first method executed is not onCreate(), but attachBaseContext(), which is overloaded by the wrapper. There, libmd is initialized and loadDexWithFixedkeyInThread() is called to process the encrypted resources. (Other methods and classes refer to more decryption routines, but they are unused remnants in our test app. ¹)

Application.attachBaseContext() override is the true entry-point of a protected app

Calling into the native file for decryption and swapping

The rest of the “app” object are simple proxy overrides for the Application object. The overrides will call into the original application’s Application object, if there was one to begin with (which was not the case for our test app.)

Proxy stubs to the original Application’s object

The remaining components of the DEX file are:

Setters and getters via reflection to retrieve system data such as package information, as well as stitch back the original app after it’s been swapped in to memory by the native component.
The main activity com.seworks.medusah.MainActivity, used to start the original app main activity and collect errors reported by the native component.

Native Component

The protected app shipped with 3 native libraries, compiled for ARM and ARM v7. (That means the app cannot run on systems not supporting these ABIs.) We will focus on the core decryption methods only.

As seen above, the decryption code is called via:

m = new MedusahDex().LoadDexWithFixedkeyInThread(
    getApplicationInfo(), getAssets(),
    getClassLoader(), getBaseContext(),
    getPackageName(), mHandler);

Briefly, this routine does the following:

Retrieve the “high_resolution.png” asset using the native Assets manager
Decrypt and generate paths relative to the application
- Permission bits are modified in an attempt to prevent debuggers and other tools (such as run-as) to access the application folder in /data/data
Decrypt and decompress the original application’s DEX file resource
- The encryption scheme is the well-known RC4 algorithm
- The compression method is the lesser-known, but lightning fast LZ4
- More about the decryption key below
The original DEX file is then dumped to disk, before the next stage takes place (dex2oat’ing, irrelevant in the context of this post)
The DEX file is eventually discarded from disk

Retrieving the decryption key statically appears to be quite difficult, as it is derived from the hash of various inputs, application-specific data bits, as well as a hard-coded string within libmd.so. It is unclear if this string is randomly inserted during the APK protection process, on the server side; verifying this would require multiple protected versions of the same app, which we do not have.

ARM breakpoint on DecryptFileWithFixedKey(), step over, and destination file in r6

A dynamic approach is better suited. Using JEB, we can simply set a breakpoint right after the decryption routine, retrieve the original DEX file from disk, and terminate the app.

The native code is fairly standard. A couple of routines have been flattened (control-flow graph flattening) using llvm-obfuscator. Nothing notable, aside from their unconventional use of an asymmetric cipher to further obscure the creation of various small strings. See below for more details, or skip to the demo video directly.

Technical note: a simple example of white-box cryptography

The md library makes use of various encryption routines. A relatively interesting custom encryption routine uses RSA in an unconventional way. Given phi(n) [abbreviated phi] and the public exponent e, the method brute-forces the private exponent d, given that:
d x e = 1 (mod phi)

phi is picked small (20) making the discovery of d easy (3).

Find d given phi and e, then decrypt using (d, n)

Decryption (p = c^d (mod n)) … with a twist

The above is a simple example of white-box cryptography, in which the decryption keys are obscured and the algorithm customized and used unconventionally. At the end of the day, none of it matters though: if the original application’s code is not protected, it – or part of it – will exist in plain text at some point during the process lifetime.

Demo

The following short video shows how to use the Dalvik and ARM debuggers to retrieve the original DEX file.

This task can be easily automated by using the JEB debuggers API. Have a look at this other blog post to get started on using the API.

Conclusion

The Jar file aj_s.jar contains the original DEX file with a couple of additions, neatly stored in a separate package, meant to monitor the app while it is running – those have not been thoroughly investigated.

Overall, while the techniques used (anti-debugging tricks, white box cryptography) can delay reverse engineering, the original bytecode could be recovered. Due to the limited scope of this post, focusing on a single simple application, we cannot definitively assert that the protector is broken. However, the nature of the protector itself points to its fundamental weakness: a wrapper, even a sophisticated one, remains a wrapper.

The protector’s bytecode and native components could use a serious clean-up though, debugging symbols and unused code were left out at the time of this analysis. ↩

Crypto Monitoring with the Android Debuggers API

Updated on May 4: JEB 2.2.3 is out. All users can now use the Android debugger modules.

In this short post, we will show how the debuggers API can be used to monitor an app execution, hook into various key methods and classes of the standard Java cryptography SPI, and extract input and output data, as they flow in and out encryption/decryption routines.

Very handy to retrieve encrypted data used within an app or exchanged with a remote server. ¹ Check out the following video to see what we are talking about:

The sample code of the AndroidCryptoHook plugin can be found on our public GitHub repository.

This simple plugin does the following:

It looks for an active Dalvik debugging session
It sets up a debugger listener, which will listen for BREAKPOINT and BREAKPOINT_FUNCTION_EXIT events
It currently “hooks” 3 methods of the javax.crypto.Cipher abstract class:
- byte[] doFinal(byte[] input)
- int doFinal (byte[] output, int outputOffset)
- int update(byte[] input, int inputOffset, int inputLen, byte[] output)
When any of the hooked method is called, the associated hook onEntry method is executed, which will dump interesting input parameters
When the same hooked method returns, the associated hook onExit method is executed, which will dump interesting exit parameters and return value

The hook here consists of a double breakpoint, one triggered when a method is entered, another one, when it exits.

A hook on doFinal() capturing plain text data just before it gets encrypted

The code for that Java plugin is fairly simple. More hooks could be easily added, and hooks in native libraries could be set up in a similar fashion. Lastly, always keep in mind that the API in general (and this plugin in particular) can be leveraged by UI or headless clients. Automate things away if you need to.

The one and only entry-point for developer resources is our Developer Portal. Do not hesitate to reach out, publicly or privately, if you have issues or pointed questions. Thank you.

Dynamic execution monitoring can be achieved in several ways. Debugging a target is one of them. ↩

An introduction to JEB Android Debuggers

=> Updated July 10, 2019
=> Other blog posts in the series:
– Debugging Android apps on Android Pie and above
– Crypto Monitoring with the Android Debuggers API
– Debugging Dynamically Loaded DEX Bytecode Files
– Advanced Debugger Usage via the Interpreter

Dalvik and native code debugging has been supported in JEB since version 2.2:

Linux ELF and Windows PE code object support (32-bit and 64-bit)
Disassemblers for Intel x86 and x86-64, ARM 32-bit (including floating point support), ARM 64-bit (Aarch64) and MIPS.
Debuggers for Android Dalvik virtual machines and native Android processes

This post presents the details and caveats pertaining to the Android debuggers, shows how to debug APK’s Dalvik and native code, and explains the debuggers limitations.

An on-going debugging session of Android Dalvik and native ARM code.

Introduction

Debugging closed-source Android apps has historically been challenging at best, close to impossible at worst. JEB took a stab at solving this problem by abstracting away a wealth of low-level details and caveats related to debugging so that analysts can focus on the Dalvik code and associated decompiled Java source, as well as native code.

The Android debuggers make the task of reverse-engineering complex apps, e.g. those using a mix and bytecode and machine code, finally possible in practice. With the rise of app protectors and obfuscators, support for full-scale debugging has become more and more pressing. Earlier in February, we published a video highlighting one major feature of these debuggers: the ability to seamlessly debug to-and-from Dalvik and native code. We will explain in details how to use the features highlighted in the video.

Another area we will explore is the debugging API. The debuggers abide to the JEB IDebuggerUnit family set of interface. They can be used to automate debugging tasks, and allow for easy integration in analysis pipelines.

Requirements

The JEB Android debuggers run on all JEB-supported platforms (Windows, Linux, macOS). Please verify the following before attempting to start a debugging session:

Make sure to have the Android SDK installed. Ideally, you also want to have either ANDROID_SDK_ROOT or ANDROID_SDK environment variable pointing to the SDK folder.
Enable Developer options and allow USB debugging on the intended physical target device. (Debugging is enabled by default on the emulators.) On physical devices running Android 4.2 and above, one way to make sure of that is to run the adb devices command. If the device is shown as unauthorized, a pop-up on your phone will appear to request authorization.

Debugging non-debuggable apps

Normally, only apps whose Android Manifest explicitly has a debuggable flag set to true are debuggable. However, this is rarely the case when analyzing in-the-wild applications, malicious or otherwise. In such cases, you have several options.

Run the app in an emulator. Emulators have the ro.debuggable property set to 1. This means they will debug all apps, regardless of the debuggable flag in the Manifest.
Use a rooted phone. A rooted phone will allow you to modify the ro.debuggable property, and change it from 0 (standard on production devices) to 1. The rooting process is out-of-scope for this document: it is device specific and rooting instructions can easily be found online. As for ro.debuggable, we will explain how to change this system property in a separate blog entry.
Unpack/modify/repack your app. (Update: JEB can do this for you) Depending on whether the Manifest is obfuscated or not, this may be the simplest option. If the Manifest is obfuscated, the repacking process may not work. Another caveat applies: signing. You will have to sign the repacked app using your own key; be aware of the implications if you choose that option. ¹

Caveat: Native code in non-debuggable apps

When it comes to debugging native code of non-debuggable apps on a rooted phone or emulator, other limitations apply. ² JEB tries its best at abstracting them away. However, things might be unstable depending on which phone and OS is being used. Do not hesitate to let us know if you encounter issues.

Note that most of our tests are done on Nexus devices running vanilla Android 5.1 and 6.0. Using similar devices for debugging will likely reduce the chances of running into corner-case problematic situations.

Starting a debugging session

Currently, JEB can start Android debugging sessions only when analyzing APK files. If your main artifact in JEB is an orphan DEX file, the UI client will refuse to start a debugging session.

First, retrieve your target APK and get the app ready for debugging:

Make sure the APK matches the one that will be executed on the target phone. You can download the APK using adb ³:
- adb shell pm list packages -f to retrieve a list of packages and the associated path to APK
- adb pull <pathToAPK> to download the APK
Start the app on the phone
- Via the App Launcher for instance, if attaching to an already running app is an acceptable scenario
- If you want the app to wait for the debuggers to attach to it before it starts executing any code, you can run something like: adb shell am start -D -S -n <packageName>/<activityName>
  - A popup will be displayed on the phone, indicating it is waiting for a debugger to attach to the VM

Second, in the RCP desktop client:

Start an analysis of the APK file
Open a view of a main DEX file
Once the focus is on the DEX view, open the Debugger menu, and click on Start…

The Debugger/Start, used to start or attach a debugger, is available once the code view of a support Code unit has the focus. Here, the focus was on Dalvik bytecode.

In the Attach dialog window:

Select the target phone and the target process that matches your app, and click Attach.
Unless you tick the “Suspend all threads”, The app will be immediately be run/resumed after attaching.
The process filter is normally filled out with the APK package name. Simply press enter to filter out entries.
(No longer the case) Your entry must have a D flag. This flag indicates that the target device will accept incoming debugger-attach requests to the target process. If you are trying to attach to an entry that does not have this flag, the operation will fail.

After attaching, the app, you should see one or two additional nodes in the Project tree view.
- If the app does not contain native code: there will be one node, representing the Dalvik VM debugger
- If the app contains native libraries (*.so files in lib/ folders): there will be an additional node to represent the native code debugger
When a debugger is successfully attached, the corresponding node has a light green background.

Example: Two debugger nodes (VM, Process) currently not yet attached to the target

Example: One debugger node (Dalvik only) currently attached to the target

Views and layout

Open the VM debugger views by double-clicking the VM unit node. At this point, you will want to customize your layout: debugger views can seriously clutter the workspace area. See an example of customized layout below:

This customized layout shows: – the code hierarchies (Dalvik, Native) in the lower left corner – the VM debugger views stacked on the top right corner – the Process debugger views stacked on the lower right corner – the Console and Logs at the bottom

Layouts can be customized via the Window menu; more details can be found in a previous blog entry.

The debuggers should now be attached.

The Process debugger is never paused after attaching
The VM debugger is paused if and only if the “suspend threads” option box was ticked

Keep in mind that pausing the Process debugger (ie, suspending the native threads) will obviously freeze the higher-level Dalvik VM!

Next up, let’s review the debugger controls and controls.

Basic debugger controls via the UI

Active debugger

The most important thing to remember about debugger controls is that the UI controls affect the debugger related to the view currently focused.

Unlike most tools, JEB allows multiple debuggers and debugging sessions to take place at once. Therefore, be mindful of which debugger is being controlled when pressing menu entries or toolbar buttons: If the focus is within the DEX view or any VM debugger view, the controls are connected to the VM debugger; if the focus is within a code view connected to the Process debugger, the controls are connected to the Process debugger.

Controls

Basic debugger controls can be accessed via the Debugger menu or the toolbar area. They allow:

Attaching, detaching, terminating the process
Pausing and resuming the process and, possibly, its individual threads
Stepping (into, over, out of)
Toggling execution breakpoints ⁴

The toolbar contains a subset of the most common and useful controls, that are also accessible via the Debugger menu.

Not all controls can or are implemented for all debuggers. Currently for instance, pausing individual threads of the Process debugger is not possible. When a control is not available, depending on which control it is and the severity of the failed operation, the user may be unable to activate it (eg, grayed button), receive an error in the logger, or receive a pop-up error in the client.

Breakpoints can be set/unset using the handy Control+B (or Command+B) shortcut. An icon is displayed in the left vertical bar of a code view to represent enabled/disabled breakpoints .

One enabled and one disabled breakpoints.

Debugger views

Here are some of the views rendered by the UI client when visualizing a debugger unit. (Other views were added over time.)

The Threads view displays thread identifiers, status (running, suspended, waiting, etc.) as well as the stackframes when a thread is paused. Depending on the target processor, there may be one or more stackframes, showing the location (program counter register or address) of the current thread.
The Breakpoints view displays active and inactive code breakpoints. (More on breakpoints and breakpoint types later.)
The Locals view shows the generic variables registers. They can be virtual slots of a VM, registers of a native process, complex variables inferred by the decompiler, etc.

Debugger views of a paused Dalvik VM. Stackframes are visible, along with some local variables.

Every debugger has specifics that are relevant to the target being debugged. While the JEB API and front-end are trying to abstract the nitty-gritty details away, there are times when generic controls are not enough. In the next section, we discuss how users can issue such commands via the debugger console.

In the case of the Dalvik VM, the Locals view can be used to display complex objects or arrays, as is shown below:

Variables view of the top-level frame of a suspended Dalvik thread

In the case of local variables, the type of a Dalvik slot (v0, v1, etc. ) is usually inferred thanks to the Dalvik decompiler. A JEB build that does not ship with the decompiler will not be able to display most frame variables accurately.

Live variables overlays

When a thread is paused, the debuggers (native as well as Dalvik’s) provide overlay information when the mouse cursor hovers over registers, variables, class fields, or any other visual element that holds data.

In the case of the Dalvik debugger, overlays also work in Java decompiled views.

Advanced controls via the console

The debugger units make use of the IUnit.getCommandInterpreter method to provide clients with command interpreters to execute advanced debugger commands, that may not be readily made available by graphical clients.

In the UI client, command interpreters offered by units are accessible via the Console tab. Once the Android debuggers are attached, switch over to the Console view, and type list. This command will list all command interpreters currently attached to the console:

Two interpreters are made available, one by the VM debugger, another one by the Process debugger

An interpreter has a numeric id as well as the name of the unit that created it. Switch to an interpreter with the use <id|name> command.

Switching to the interpreter connected to the Process debugger

The special command help, available in all interpreter contexts, lists all commands made available by the interpreter currently in use.

Functions provided by the Process debugger interpreter

In this example, we can see that the Process debugger offers ways to read and write to memory, set registers, and also issue low-level GDB commands (use this option carefully).

Settings

The Android debuggers offer options to control low-level debugger parameters, such as ports and timeouts.

If you wish to disable native debuggers entirely, set the DoNotUseNativeDebugger to true.

API for Scripting and Automation

Debugger modules in JEB implement the set of interfaces contained in the com.pnfsoftware.jeb.core.units.code.debug public package. The principal interface in this package is IDebuggerUnit. Plugins, scripts, or third-party clients wishing to automate the usage of debuggers can us these well-defined interfaces. The official UI client uses this public API. Anything that the UI client does (and more) can be done and/or automated by third-party code.

Check out our post on Android crypto primitives hooking to see how the API can be used to retrieve pre-encryption or post-decryption data on the fly.

Within the next couple of weeks, we will upload sample code on our GitHub repository demonstrating how to use the JEB Debugger API.

Sample code making use the of the JEB Debugger API

More on scripting: https://www.pnfsoftware.com/blog/crypto-monitoring-android-debuggers-api/

-nicolas

A technical implication is that apps performing health checks such as signature verification can easily detect that they have been signed by an unauthorized key. But then again, running an app on a rooted phone or an emulator is also something easily detectable. Each method has its advantages and shortcomings, be aware of them. ↩
They mostly have to do with the run-as Android utility. JEB ships with a modified version of the utility to allow debugging the native code part of non-debuggable apps. ↩
We strongly recommend our users to get familiar with the Android system tools and debugging tools ecosystem, in particular adb, am, and pm. ↩
Toggling breakpoints on and off is currently not available in decompiled views. ↩