Your App, your public API
If you have an App in the market, you already have a public API.
And getting access to that API is much easier than you might think. I am not a security expert, and I have never done any kind of reverse engineering.
Motivation
I had one of those weekends where I wanted to gather some information from a webpage for a short personal project, and I checked if the web page provided some kind of API. It actually did, but, you had to request access filling a form, that someone would review, and then provide access to the API.
I did not want to wait, so it was a good excuse to try to extract the API from their app.
About APIs for mobile devices
Mobile API are exposed to everyone. With more or less trouble, but your API contract can be extracted.
** You can not easily deprecate the API ** : Once you have released an App you can not deprecate that API from one day to the next. The point is that, releasing a new version of the App does not mean that your users will update it soon (some of them won’t update it at all).
Your mobile app is a frontend, that is easily “hackable”. Do not expect your mobile devs to fix API deficiencies.
** It could be your presentation card **. Are you looking to hire developers? candidates can know a lot about your code by looking at your APIs.
Example: VIBBO
Vibbo is a marketplace, where people can sell and buy used items. It is like LetGo or Wallapop, with the only difference that they are not using HTTPS for their main webpage (ehem! :P).
I did not choose this app for any particular reason: I selected a random downloaded app from my phone.
I used the Android App (is easier to extract the app when we need to decompile it). The same process could be done on iOS, but with different tools. Look at the references at the end of the post. (Hint: if the man in the middle is not enough you will need a jailbroken device).
I did not extract all API calls, I only wanted to show how it can be done.
Set up the MITM
First of all I connected the phone to the same network than my laptop, and looked for a tool to help me do the Man-in-the-middle.
There are several tools to inspect traffic, but I decided to use MITMProxy, mainly because:
- It provides a very easy way to install self-signed certificate. (You run the proxy in interactive mode, configure your device to use that proxy, visit http://mitm.me, and follow the instructions).
- It has an “interactive” mode (mitmproxy) where you can modify requests on the fly, and a “dump” mode (mitmdump) to save requests flows to a file so you can later and reproduce the calls as many times as you want.
- An easy way to Filter the requests. (Take into account that once you set the proxy all the requests from the device will use that proxy configuration).
- It is scriptable with Python (actually all the tool is written in python)
It has a lot of options, I recommend you to read the full MITMProxy documentation.
Key bindings are pretty similar to those used in the Vi editor.
Capture all traffic
I used mitmdump for the first run, so I saved the request / response flows to a file, so after that I load the flows from the file with mitmproxy to inspect everything.
mitmdump -w vibbo.mitm
If you capture before downloading the app from the Android market, you will have the .apk file inside the capture file (that can be useful later, to dump, extract and decompile it).
Once we finished some navigation, I closed the app so I could start working with the file using mitmproxy (the interactive mitm tool):
mitmproxy -nr vibbo.mitm
(the n flag is to not run the proxy, otherwise the proxy would read the captured saved file, but also would keep capturing new traffic).
Then I could have a look at the domains I was interested (because I did not have any filtering yet, I could see all traffic, not only from the App we are inspecting, but also all other apps in the device).
I chose to inspect domains containing the ‘vibbo’ string:
mitmproxy -nr vibbo.mitm -f '~d .*vibbo.*' ```
At first saw I could find different kind of apis:
-
v1 via HTTP
-
v2 via HTTP
-
the users api call using HTTPS
When I looked at some of the requests, I found that the query parameters were encoded. The q param looked like a base64 encoded string, while the s param looked like it was hex encoded.
I tried to decode the q param, to find that It did not have any meaning to me:
Looking at the response, it was not much better:
C3cUjJrZeKy/hFeBHzn2UHllH2edGisyn6TBew82LKN0XsMaS7DcNYh8Y3PPAzdqBGpzKqhQoNUt8XwMBp+tHCs5Emky74bHje4Dp+JMdIg9NdLsexz6NzSpTguDQBwjSC1lh8PPZcTiMCbkjcUbF3bPqBIlLnDDnvbCElzTQRk6hUlvZhFamppBGYcXLCUk4nn6mrbU50lMpRCDkKq/3svfR2oRWxB0inNecQ984RNXQ2VT9H73VRZ1PL2MRSK+8C7gFNL9uAhbjErfpfzMnIg4lXXbkzrTGlomxRJ8I8pAE3dTctd58oujGchWb76FAT0l3Gqj0yWHmB460s0KDB0Ip8atZWT+UQ9HEk29uec6s+QxZTMSiPVg4idGQjLdiYTL4nGyVRTp5r3Mczc/q4jdADcBgwmTjtux9hAgNEFpc4nBHYTd4qXAxLQZ7SOgNK8RHWBparzpG5YKegvDtGUad/p2SwScKOmItNh5mBwoC63Ra4NvyffG0Ig292nzP/D8sHC5CvF1jSE4a9s+GUNQVePNLDZyfrHsACkNnKyPPCR2+DbPGgmmHT/rXrr6J1sfms/LIFpQGvZ3dKuVfMYNLsJnFXYN1XRn/dotH50ZCDFGkdVfGOV1jM3anKIX1WHqDGDJCZSJJ9oTH+tI1tt+pb/olJhsfgUQwmD0z5/v3E4vvp+deZtbTwKwCe97vBjDiC3FyUxUObQqCfKHVqJatEIdkWfO3B0DC2QJ9huym8NcgoH1UaIcRLa8wLBHKVdTyy+6+LClZpdZ5UEyMiBfA/MB+aaCPUqGJzXeATe4Yh2CbGnIUitmTZEv6p+1tPRiwHLXZkED0h++WaM32otlk0qOtEUrd9zISStI54tYNmnfscQZ8RJqp7CaSuVoSzzqzl68MH3Lup758IUxZoGV+dj0OQwkwjyivmpSp6W406It5czcED8w7FsJ433dmXMtFCxyKx9Q4RKwBtsm2g==
So, I thought I would probably need to access the app code to see how the encoding was done.
There were other requests (_ http://api.vibbo.com/API/forms/search?auth_token= _), that did not had encoding. In this case, the response looked like a set of categories and rules to show suggestions for the app’s search view.
I was missing something. I could see the /detail request, but not the actual search that I performed before (for PS4 consoles). So I loaded again the file, this time without filtering by domain name, to find that I actually missed an important request that was not being done against a vibbo domain:
http://hl-search.scm-pro.schibsted.io/mixed-ads?q=ps4&search_by_map=true&latitude=41.3850639&longitude=2.1734035&distance=4000&category_parents=15&price=100%3A400&sort=distance%2C-publish_date%2Cprice
and that it looked that were the actual search result
There were also api calls to get information about a user, following an easier RESTful scheme, with non-encoded params in the url, and also a non encoded response.
GET https://ms-vibbo--app-userapi.spain.schibsted.io/v1/accounts/16895146
From my point of view, it looks like the encoding part of the /detail API endpoints are a measure to compensate for the use of HTTP instead of HTTPS.
But I wanted to get access to this endpoint too, so, the next step was to decompile the app to check how to decode the requests.
Decompiling the APP
First of all, I extracted the apk (the body of the request is actually a gzip, once you decompress it you get the .apk file), and decompressed it (the .apk is actually a zip file).
Then I found the .dex files, and I used the dex2jar tool te convert all .dex files into .jar files, that I fed to the Procyon decompiler.
Procyon can choke with some methods (where it spits out the Java bytecode opcodes), but it does a very good job producing Java source code.
procyon-decompiler classes-dex2jar.java -o .
procyon-decompiler classes2-dex2jar.java -o .
And it looked that the source code had been obfuscated probably with proguard.
So, I did a simple grep to search for the api.vibbo and /detail strings, so I could find spots to start diving into the code. And I found a couple of definitions:
Constants.d = "http://api.vibbo.com"
Constants.H = "/api/detail"
After looking around a little bit for the constants, and not finding anything in the code that could easily hint me how to encode / decoded the fields, I decided to grep for Base64. Just before the code that performs the base 64 encoding I should find the encryption part for the query param and the response body.
And finally found two interesting classes:
- com.anuntis.segundamano.api.ApiHandler : This class gives us information about how the s param is baked: by making an md5 over the q param before the q param is cyphered:
public static InputStream a(String s) {
final InputStream inputStream = null;
try {
final Encryption encryption = new Encryption();
final String encryptAsBase64 = encryption.encryptAsBase64(s);
s = encryption.md5(s);
s = String.format(Constants.h, URLEncoder.encode(encryptAsBase64, "ISO-8859-15"), s);
final Response execute = new OkHttpClient().newCall(new Request$Builder().url(s).get().addHeader("charset", "iso-8859-15").addHeader("User-Agent", "android").build()).execute();
InputStream byteStream = inputStream;
if (execute != null) {
byteStream = execute.body().byteStream();
}
return byteStream;
}
catch (Exception ex) {
ex.printStackTrace();
return null;
}
}
- com.anuntis.segundamano.utils.Encryption : This is where the main encryption / decription algorithm is performed.
Encryption
The encryption class required some patience, since the decompiler failed to transform byte code into java code.
private static String getPass() {
return "4425363b0c9ecd76a4fee7680e41e3285ef507e18521a916ea6bd9f589df89ef2c2fda41547bd32543b38e82bcbc74da";
}
public String decrypt(final String p0) {
//
// This method could not be decompiled.
//
// Original Bytecode:
//
// 0: aload_1
// 1: invokevirtual java/lang/String.length:()I
// 4: bipush 16
// 6: irem
// 7: ifne 54
// 10: ldc "AES/CBC/NoPadding"
// 12: invokestatic javax/crypto/Cipher.getInstance:(Ljava/lang/String;)Ljavax/crypto/Cipher;
// 15: astore_2
// 16: aload_1
// 17: ldc "ISO-8859-15"
// 19: invokevirtual java/lang/String.getBytes:(Ljava/lang/String;)[B
// 22: invokestatic com/anuntis/segundamano/utils/Base64.decode:([B)[B
// 25: astore_3
// 26: aload_2
// 27: iconst_2
// 28: getstatic com/anuntis/segundamano/utils/Encryption.secretKey:Ljavax/crypto/SecretKey;
// 31: getstatic com/anuntis/segundamano/utils/Encryption.ivParameterSpec:Ljavax/crypto/spec/IvParameterSpec;
// 34: invokevirtual javax/crypto/Cipher.init:(ILjava/security/Key;Ljava/security/spec/AlgorithmParameterSpec;)V
// 37: new Ljava/lang/String;
// 40: dup
// 41: aload_2
// 42: aload_3
// 43: invokevirtual javax/crypto/Cipher.doFinal:([B)[B
// 46: ldc "ISO-8859-15"
// 48: invokespecial java/lang/String.<init>:([BLjava/lang/String;)V
// 51: astore_2
// 52: aload_2
// 53: areturn
// 54: ldc "AES/CBC/PKCS7Padding"
// 56: invokestatic javax/crypto/Cipher.getInstance:(Ljava/lang/String;)Ljavax/crypto/Cipher;
// 59: astore_2
// 60: goto 16
// 63: astore_2
// 64: aload_2
// 65: invokevirtual java/security/NoSuchAlgorithmException.printStackTrace:()V
// 68: aconst_null
// 69: astore_2
// 70: goto 16
The byte code was actually an AES encryption algorithm. And I found that in the class there was a getPass() method, but actually that ‘pass’ was used to generate the the secret key and initialization vector for the AES algorithm
public Encryption() m{
this.passphrase = getPass();
final byte[] hexStringToByteArray = this.hexStringToByteArray(this.passphrase);
final byte[] array = new byte[32];
final byte[] array2 = new byte[16];
while (true) {
try {
System.arraycopy(hexStringToByteArray, 0, array, 0, 32);
System.arraycopy(hexStringToByteArray, 32, array2, 0, 16);
Encryption.ivParameterSpec = new IvParameterSpec(array2);
Encryption.secretKey = new SecretKeySpec(array, "AES");
}
catch (Exception ex) {
ex.printStackTrace();
continue;
}
break;
}
}
Given that I only needed to check if that was what I required to be able to use the /detail endpoint. So I coded a python script, and fed it the q param, and a response from the captured request.
# -*- encoding: utf8 -*-
# from __future__ import unicode_strings
import requests
import json
from pprint import pprint as _p
from datetime import datetime
import io
import base64
import hashlib
import random
import struct
from Crypto.Cipher import AES
class Encryption(object):
"""
to view the algorithm I used the jvm opcodes:
https://en.wikipedia.org/wiki/Java_bytecode_instruction_listings
"""
def __init__(self, key, initialization_vector=None):
self.key = buffer(key)
if initialization_vector:
self.initialization_vector = buffer(initialization_vector)
else:
self.initialization_vector = buffer(bytearray(16))
self.cbc_block_size = 16
def _pkcs_7(self, message, blocksize):
padd_byte = blocksize - (len(message) % blocksize)
padded_message = message + struct.pack("{}B".format(padd_byte), *[padd_byte]*padd_byte)
return padded_message
def encrypt(self, message):
if len(message) % self.cbc_block_size != 0:
message = self._pkcs_7(message, self.cbc_block_size)
aes_crypt = AES.new(self.key, AES.MODE_CBC, self.initialization_vector)
res = aes_crypt.encrypt(message)
return res
def decrypt(self, message):
aes_crypt = AES.new(self.key, AES.MODE_CBC, self.initialization_vector)
message = aes_crypt.decrypt(buffer(message))
return message
def encrypt_ftcws(self, message):
padded_message = self._pkcs_7(message, self.cbc_block_size)
aes_crypt = AES.new(self.key, AES.MODE_CBC, self.initialization_vector)
res = aes_crypt.encrypt(message)
return res
def encrypt_to_hex(self, message):
enc_message = self.encrypt(message)
hex_message = ''.join('{:02x}'.format(x) for x in bytearray(enc_message))
return hex_message
def decrypt_from_hex(self, message):
message = bytearray.fromhex(message)
return self.decrypt(message)
def encrypt_to_b64(self, message):
enc_message = self.encrypt(message)
# b64_enc_message = base64.b64encode(enc_message)
b64_enc_message = ''.join('{:02x}'.format(x) for x in bytearray(enc_message))
return b64_enc_message
def decrypt_from_b64(self, message):
decoded_b64 = base64.b64decode(message)
return self.decrypt(str(decoded_b64))
def vibbo_test(encoded_snippet):
passphrase = "4425363b0c9ecd76a4fee7680e41e3285ef507e18521a916ea6bd9f589df89ef2c2fda41547bd32543b38e82bcbc74da"
bin_passphrase = bytearray.fromhex(passphrase)
secret_key = bin_passphrase[:32]
initialization_vector = bin_passphrase[32:]
enc = Encryption(key=secret_key,
initialization_vector=initialization_vector)
qparams = str(enc.decrypt_from_b64(encoded_snippet)).encode('ascii')
endstr = -1
while ord(qparams[endstr]) < 16:
endstr -= 1
qparams = qparams[:endstr+1]
try:
jparams = json.loads(qparams)
_pp(jparams)
except Exception as e:
print(qparams)
test_a="GenYvIyueH+CY1or7Q4AhM5fKB0qF2jr6qkqn6UKllF0BpSeHIAFEAiieRHQ36/PBc7oLZaOBy+s1iQYgOX/bxE/SKjzEOi5CQ5FqncNRrA1extJJdJVfutswgS+WA9vepzO2dRnKvRNaS+OqIkoQLD5AA+AhX4qPlew18yzoPtkP14cqM3bIVcRN3r9M6TVTzGEg9QZL4Ow5qBXvYBLg3VBixYHQku2k6Z5mlwAVBabQTHHg0WKrP/5sEG5A0+K541HQZvqS4wCIMBzfWojQkDazBt6IOLMez79KW0JaeUkSKf/DTtao0L6J+Y4LTTWpU6ti8XEtp6lWwwgkhIGF349HhtI3ESDHDOo/fZJ9fUDtXzTHy10VSVhqvEt1DprLZ//Nzk+NdIoKKJQpDJf7xXjsUD8C2o6CXvf8/U09vaSB7e/75SvnfKdwCdKJ/dyhp3XKeVBkwZE0m5JiDzRauWeEkuTjnm4V8PcwIN0kUcAFfnT1738cu14SvvSRSaJMW2gXqrohCRRS9zk18V4np/ojFKvrp+W36Z3MDCAXtFhe0KcIBkXXtDZdLomgi5+5MnODVj0oVSL4yeV/WoVVkXlmq0u1H13CozMQEp5RQphOPt02EmMW2HWhY+zj/KKSMzh2VA8p0h+anbmjw0goEpl6jwmOVDDv5QxZpwX/y3uYQoFjy++vMqG8IbjJ33MdLMwJKq8rCdrDS/aY5Mj/k0BVbmyMNgyvOJb7+DEmxRbla3oqTqEnAWfFetGM9kAHqe1xK+AblNct5WIEjgxPEMRgtrG+/D2RArKlwK5+MWBBc7T+vprrLeXjnvdC8E9h4wOhUek53+M7R4Vz3YWDnXiFWGCBDnIZsHEvd6QBC3qn22P5IB2h5hdp6wjuTaNCAK3joQMq6xC9SYiBBBXcw39Anndxu67h1LPPMaoSjNQZre+qqj2M3hnvoy8f4o8nc4dZOyhxjomsio75R2dsEbkSvPUEjKJAAnJDw12OGIPo7T+GTdaWR2zsJKMMuSf80RT+0O2etjyqCew4LCKS3V0KFHawlgXSmvuqzo7G/sbCUnxUQDwGYa8esE289GbLgZn7WiWG88cchCG+mehLXxJCp4qHg5ppp2P6uhW73d84FRKVgGFJf3F+PT7oAR3h2eTAfXLKvRz8RiTD6yruZWhqafDTm3oiyTT6zvXlJYDXTMkDLCw3EVGfCLEpaXDCTotlEMmqHwv5ZjtkO6hMp5TRqbszayP+vTVr21/upEvXtSAQSAma/ZMYHbmlnR0cYe8pXbpqEwS/rvL/n3jwbA/j7R8aGIBemkhH6M8YJVO7tdO7kzi/heSB2w4O/KkQkJtXsE5NZN4Ts167b+FRRyyvoY185qC3eV/7bCZrV7v4+VfVcOYe2zy2guqihWv3yaXns+OSeESIofcTjBPSiDwrLpsS1LMCh9INFdwMJmvjuz5oARh0YC9ZRuDThzuJUKqugdTxDuEVYQmKlZkei6w5u0Pdky6mm9XlOwEISLX9jQn4VNToSxlKxPwGtAMq8ikXCQDriZ4n8Mk43ntZw5qKKbAWgNZOxXsepQNmTKChPD+y9L59LtSXGT7KuKfH0Z6DJSMYzyYcViIruYV9Pmit83rUi9OMspjr9gJM/i4EKzFbOfIitLizr4wLFDATOvdI4ZZizssJj5e44QxoIidHV/GC0sWaU/ZkulHAMD3Xx2BPq1t2+ULctp1BzbK+AIB/ldhGHWj3lkQUq5txxqDif04w/XblVRsUCMhy/XRyNAvjgCBN9D5vs/VbCpHXuH/b6hCXOv/q/ar/J/yZmco4Etp8xFo707FaEaly2BemDgA75sX4vvq+V5Io6tmpFe/KgAEqDEJ2T8FAO3pyB+KM5b+q3HZgGZqOBMIQD7PBP99/V2eIh+OWY3qGTmIQ7v19jcShk3QJ1o2wWulCDQAfxrO7aXIfmSMtJinuxwMS6bf43J7ZG1NVGLbb8aH1kfn28awEjdYkPmRjzWh6bGcYSvU1mVNT4G8kFisndO3dxjaE8cCT93nroYTf/ZINiXhP3kpki4ltIJj8GThIA=="
test_b = "FxLN30qJl+zcQfcXh4EMAA=="
test_c="C3cUjJrZeKy/hFeBHzn2UDkH8imDgOo8rl2ma8tldnAt3IOUrX0f7PsNdhU7bHR0swqI4gnGBCpVRoamH6zJQ/HlP69v6CiVZ6iDECqEZjgetOsZyeDMncXP1WF5YbnaEOCyGApjTYIeX4d/zwOpJSEsA0xK2NUTKswyIaYdfuIfaKSTbTc96x6hxpUrGs2RUPiJhtMR+Zq9ntbRqe+XWaJyzT1od2MWduqA0Le77xB+ASbe0c46kHK+o+f6rmYp9DQgvT4BeqVTHbwC1V74TQ6kdr/Xb/ORhmupYSHy3T00Fck0tqzfSvojVJSlWosHyLgo5+cj8JDOU1PNOBIcrDhtSFj4kMtqBTXPGqqof70amNrR8Fm1Xuuum7CWPavsvh7NwwASi7J0OBHNxXTTwYwBqPf+uXdvcQV9/QK3A2WZjpK7QXdIhc2JQxDYfy17T0TdQGIehfOussGTfOZhg8Ez0OBNAHe408Mf83mMMepgJPwH5Gdx5njyJZ+0gpa/IrvMS9R3yXjQKsvBqg2BTOJ+qTujLEAYEJlmW64P7clekHdHrB9Cs7d5/UEOZqieneh/3cCTr9A90lYXLjmZ1QKfwkSAkM5nyDdnZOE2xX+lLCpV8uh2nEP3Rwfqa8jj7B2Frf05AV/8Lq8bcOvqY0VfKRw+QrL/X/tR6kJ6ratwnxBZoGM/TGMkh8l4AqkjWE5WElEK1ugeRivynScVL7hDV1PkkRe67C9ozfJHxKybtvZIOOn0RHPyVEzC8IEIDGH0EMA5NpDHYWhndklfdtfA/oLRVOq3pnrAivSpOcEys5lJfxwMA4lKO/XrNPOZjXXtyMGkgTqfs82WFA4pxeQKQatM0cW5BXW7d6apJui2zupV4zNEEJxygkBMHGihWb9nVWdaPi3Hh8yVGU3pApgvMUG8hYS3Eqg9qiZWRBdE7uFhS4DEqeG5TIZ7XCh30vIssafliRUaUoh4G75yl9r2WjBtRS7cwvI0SmAjlNnU3z3N4y6+Gc1F+gMmkOi/H8PguDwhCOcEitQfJ79oT+bTeUPrUlpS22gL/bO66RfSQFBaoyzFoPjfG/vYU7cwkGSNmLonxRNKvp5QRDlG17uQvv32OPu2XnwPfLuBjMPzt0ebCpORc4YEdExQJMKcPoL73tSalASwqeWhzju4yg=="
vibbo_test(test_a)
vibbo_test(test_b)
vibbo_test(test_c)
Aaaaand… it worked !
So at this point I could do searches, and extract details from the results without problems.
Obviouly there are a lot of API calls that I didn’t use, but that could be ’extracted’ too.
Improvements to the API ’extraction’ process
If you plan to use the explained method, you will probably want to automate some stuff.
-
Automate decompilation Given that mitmproxy support scripts, is easy to create an script that extracts the .apk from the flows file, and automatically decompiles it.
-
Automate schema extraction: There are libraries to automate the extraction of the JSON schema, like skinfer that can be used.
Difficulting the API ’extraction’
As previously stated, the battle is mostly lost: if someone want to extract your API, will do it. However we can make it a little bit harder:
-
Using certificate pinning: But you should take into account, that If you do certificate pinning, you should provide a way to update the information of the certificate in your app (in case you have your certificate compromised, or it expires). On the other hand, the app could be decompiled, modified and then recompiled in order to tamper with the certificate check. I’m not sure what would be more time consuming, directly inspecting the decompiled code to extract API requests or doing the decompile/recompile process.
-
Obfuscate the secret key and initialization vector: We could probably try to compose / generate a complex algorithm in order to generate the secret key / initialization vector. But since we can get the code, we could run it to generate both of them.
-
Move sensitive code to a binary library. Those libraries can also be reversed, but that requires more work and knowledge than has been required for the Vibbo app. The problem is you would need to use jni, and provide binary libraries for ARM, MIPS and Intel architectures.
References
Other tools:
-
KrakenD : Can help you to hide a more powerful API that runs in the backend, or can combine information from your backend microservices.
Articles: