Usage of IccTA in Flowdroid

[yokotayokota]

Hello

I am having a hard time using Flowdroid2.9 with IccTA with the command line. As I tried to do like [#219 ], I downloaded dare and ic3-0.2.0.jar, but they don't work on my ubuntu 20.04.3 LTS. The reference site http://siis.cse.psu.edu/ has disappeared.

Is there any other way to analyse an apk using Flowdroid with IccTA? I saw the archive of soot mailing list but I could not find related subjects. I am not familiar with Flowdroid or soot, so I would be grateful if you could tell me basic instructions. Or where should I ask this question?

Thank you in advance. I look forward to hearing from you.

[StevenArzt]

IccTA has been integrated into FlowDroid, you don't need the IccTA files anymore. You only need to provide FlowDroid with the correct model, which you can generate from tools such as IC3. Have a look at the command-line parameters of the command-line application. There is -im for passing the path to the model file.

[yokotayokota]

Thank you for your quick reply. I already had a look -im command-line option but I do not know how to create the model file. I try to download ic3-0.2.0.jar but it does not work in my ubuntu 20.04.3 LTS. How should I create a correct model file for -im option?

I would appreciate it if you would help me. Thank you.

[StevenArzt]

Generating the model file is out of scope for FlowDroid, these are external tools. FlowDroid only processes the output. I do not maintain IC3, so I cannot tell you how to get a version that works on your machine. Maybe you can ask in the IC3 issue tracker at https://github.com/siis/ic3.

In fact, when we use FlowDroid in our commercial code scanner, we don't use the IC3 models. Instead of generating model files beforehand, we create the models on demand and inject them into FlowDroid. This approach relies on our static constant finder, which, however, is not open-source.

[yokotayokota]

Thank you for your answer. I understand IC3 is out of your scope and the creating models part used in a commercial product is not open-source.

Is there anyone in soot mailing list who uses FlowDroid with icc on a recent Linux? If there is, how does he/she create a model file on it? If no one can create a model file on a recent Linux or other OS, the precious -im parameter of FlowDroid will die.

Where should I ask this question?

I hope this issue would not be too much trouble for you.

[StevenArzt]

Have you asked in the issue tracker of the IC3 project to which I provided the link above? You might also ask @jacquesklein2302 for a contact that can help with issues on IC3. If I remember correctly, @MarcMil also ran the tool recently for some paper evaluation, maybe he can tell you how he did it.

[yokotayokota]

Thank you very much for your help. The IC3 project site https://github.com/siis/ic3 has not been updated for years. So I do not think I will get any reply to my question from there. I will try to ask @jacquesklein2302 and @MarcMil.

I hope I would tell you good results.

[yokotayokota]

Hello Dr. Artz, Yesterday I sended an email to Prof. Jacques Klein(@jacquesklein2302) to ask my question above, and no reply so far. I could not find the email address of @MarcMil even though I looked him up on Google Scholar and on the internet. How can I get him to notice my question?

I am sorry to bother you. I just want to use -im parameter of FlowDroid ...

[StevenArzt]

Just give them some time. I have a scheduled meeting with Jacques next week anyway and I'll see Marc the week after that. Your third option is @docteau, but I don't know whether he is still involved with IC3 (he was back in the day, though).

[yokotayokota]

Thank you for your great support. As you say, I will be good and wait. I found the contact address of @docteau, but first of all I will wait any reply before taking the 3rd option. In addition, I will try again with several other apps.

Take your time. Thank you.

[yokotayokota]

Hello. I have made some tiny progress and ran into more difficulties.

1. Like other recent Linux, my Ubuntu has Java 11 by default, and IC3 possibly does not work on it. IC3 works on Java 8.

2. All apks I built on my AndroidStudio 4.1.1 occur errors with DARE, because DARE can not handle recent DEX_FILE_MAGIC. I know DARE and IC3 are out of your scope. But I can not build even quite simple app to test FlowDroid with icc. I am wondering if I will send an e-mail to Dr.Octeau to ask about DARE. But I imagine he is not interested in DARE anymore

3. I found one apk in GooglePlay for which DARE + IC3 finished and created a model file. But I got several errors with -im parameter of FlowDroid shown at the end of this post *1.

I am stuck. Tools needed for icc have been getting older. I wonder if there is anyone who uses icc analysis with FlowDroid recently? Who should I ask for help?

* 1 Here is error messages
--------------------------------
[main] ERROR soot.jimple.toolkits.typing.fast.TypePromotionUseVisitor - Failed Typing in  at statement specialinvoke this.($u2#26): Is not cast compatible: boolean  at statement specialinvoke $u5.(com.xxxx,boolean)>(this, $u9#13): Is not cast compatible: boolean 

[yokotayokota]

Hello.

I have made some more progress. I found @JordanSamhi 's github site and there is ic3.jar that can handle apk directory. It worked with my quite simple apk, but it failed with an apk downloaded from Google Play. Even though, I do not have to struggle with Dare now.

Today I have posted a question to @JordanSamhi about the error.

I appreciate your continuous support.

[JordanSamhi]

Hi @yokotayokota,

Glad you found the IC3 version I modified, I will have a look to the issue you opened.

[yokotayokota]

Hello @StevenArzt ,

I tried @JordanSamhi 's ic3.jar but it failed with errors shown in the end of this post *1,*2. :cry:

According to him, these errors come from FlowDroid. Do you have any insight into these errors?

I attach apks that left these errors. *1 The apk is too large to attach here. 　To retreive it, follow the steps below: 　　$ tar zxvf 00_YouTube_v14.25.57.apk.tar.gz 　　$ tar zxvf 01_YouTube_v14.25.57.apk.tar.gz 　　$ cat YouTube_v14.25.57.apk_00 YouTube_v14.25.57.apk_01 > YouTube_v14.25.57.apk 　　00_YouTube_v14.25.57.apk.tar.gz, 　　01_YouTube_v14.25.57.apk.tar.gz

*2 The apk is YouTube_v16.37.36.apk.tar.gz A tiny apk that I built on Android Studio 4.1.1 left the same error. test_simple.apk.tar.gz

*1 error 
--------------------------------------
[Spark] Solution found in 24.1 seconds.
Exception in thread "main" java.lang.RuntimeException: Could not find method
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.getMethodFromHierarchyEx(AnalyzeJimpleClass.java:364)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.getMethodFromHierarchyEx(AnalyzeJimpleClass.java:363)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.getMethodFromHierarchyEx(AnalyzeJimpleClass.java:363)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeClassInterfaceCallbacks(AnalyzeJimpleClass.java:388)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeClassInterfaceCallbacks(AnalyzeJimpleClass.java:382)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeClass(AnalyzeJimpleClass.java:320)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeMethodForCallbackRegistrations(AnalyzeJimpleClass.java:245)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeRechableMethods(AnalyzeJimpleClass.java:184)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.access$200(AnalyzeJimpleClass.java:64)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass$1.internalTransform(AnalyzeJimpleClass.java:127)
        at soot.SceneTransformer.transform(SceneTransformer.java:39)
        at soot.Transform.apply(Transform.java:90)
        at soot.ScenePack.internalApply(ScenePack.java:40)
        at soot.Pack.apply(Pack.java:116)
        at edu.psu.cse.siis.ic3.SetupApplication.calculateSourcesSinksEntrypoints(SetupApplication.java:144)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:146)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:64)
        at edu.psu.cse.siis.coal.Analysis.performAnalysis(Analysis.java:73)
        at edu.psu.cse.siis.ic3.Main.main(Main.java:13)
--------------------------------------

*2 error
--------------------------------------
Warning: malformed Manifest file: action at depth 3
Error when looking for XML resource files in apk /mnt/c/Users/s.yokota/ApkProjects/YouTube_v16.37.36/APK/YouTube.apk: java.lang.RuntimeException: Unknown entry type
java.lang.RuntimeException: Unknown entry type
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readEntryTable(ARSCFileParser.java:1341)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readResourceHeader(ARSCFileParser.java:1141)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:978)
        at soot.jimple.infoflow.android.resources.ARSCFileParser$1.handleResourceFile(ARSCFileParser.java:966)
        at soot.jimple.infoflow.android.resources.AbstractResourceParser.handleAndroidResourceFiles(AbstractResourceParser.java:49)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:959)
        at edu.psu.cse.siis.ic3.SetupApplication.calculateSourcesSinksEntrypoints(SetupApplication.java:113)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:146)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:64)
        at edu.psu.cse.siis.coal.Analysis.performAnalysis(Analysis.java:73)
        at edu.psu.cse.siis.ic3.Main.main(Main.java:13)
Exception in thread "main" java.lang.RuntimeException: Unknown entry type
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readEntryTable(ARSCFileParser.java:1341)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readResourceHeader(ARSCFileParser.java:1141)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:978)
        at soot.jimple.infoflow.android.resources.ARSCFileParser$1.handleResourceFile(ARSCFileParser.java:966)
        at soot.jimple.infoflow.android.resources.AbstractResourceParser.handleAndroidResourceFiles(AbstractResourceParser.java:49)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:959)
        at edu.psu.cse.siis.ic3.SetupApplication.calculateSourcesSinksEntrypoints(SetupApplication.java:113)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:146)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:64)
        at edu.psu.cse.siis.coal.Analysis.performAnalysis(Analysis.java:73)
        at edu.psu.cse.siis.ic3.Main.main(Main.java:13)
--------------------------------------

[StevenArzt]

I ran FlowDroid on your APKs and the analysis completed without issues. However, I did not have your ICC models, so I ran the analysis without the ICCTA part. Can you supply your models?

Which version of FlowDroid are you using? The class AnalyzeJimpleClass has been removed a long while ago. Can you try again with the current develop branch of FlowDroid?

[JordanSamhi]

Hello @StevenArzt,

The problem, I think, is that ic3 relies on an old version of Flowdroid, it even itself relies on the AnalyzeJimpleClass in its code. Then I guess that IC3 should be updated to be able to take the latest Flowdroid version into account.

[StevenArzt]

Can you do this or someone from the IC3 team, @JordanSamhi ?

[JordanSamhi]

The ic3 team, I do not know but I do not think so. As for me, I could, but as I said to @yokotayokota, I have little time to invest in this at the moment, I'll see to do something.

[yokotayokota]

Dear @StevenArzt, @JordanSamhi

I am trying to rebuild ic3. I updated versions of part of pom.xm, but I can not find some jars written in pom.xml on maven repository. These jars are shown at the end of this post. I would be very glad if you told me how I should edit pom.xml of ic3? （I know Dr. Arzt is not involved in ic3, but Dr. Arzt may be concerned about the flow of this issue so I post it here.)

"infoflow"("infoflow-android") means "soot-infoflow.jar"("soot-infoflow-android.jar") in here FlowDroid github repository? Should I just download manually these two jars from here and copy them to local maven folder on my PC? ( And how can I get coal.jar?)

--------------
    <dependency>
      <groupId>edu.psu.cse.siis</groupId>
      <artifactId>coal</artifactId>
      <version>0.1.7</version>
    </dependency>
    <dependency>
      <groupId>infoflow</groupId>
      <artifactId>infoflow</artifactId>
      <version>20150607</version>
    </dependency>
    <dependency>
      <groupId>infoflow-android</groupId>
      <artifactId>infoflow-android</artifactId>
      <version>20150607</version>
    </dependency>
--------------

[JordanSamhi]

Hi @yokotayokota,

If you update the Flowdroid version in IC3, as I said, you will stumble into some problems to build ic3 since it would need some code changes. If you want coal, you can buil it from here: https://github.com/siis/coal

[yokotayokota]

Dear @JordanSamhi,

Thank you for your quick reply. To fix ic3 code seems out of my hands. I am sad. It is not my intention to interrupt you.

[JordanSamhi]

Like I told Steven, I will do something to fix ic3 with the latest versions of Flowdroid, but I can't do it at this time, I will keep you posted.

[yokotayokota]

Thank you @JordanSamhi, I am willing to wait for you.

[yokotayokota]

Dear @StevenArzt:

While waiting for @JordanSamhi, temporarily I decided to use an old version of Android Studio, and it works fine so far. I got a result file from ic3+FlowDroid2.9 -im parameter as shown at the end of this post *1. I'm happy to see that FlowDroid analyses across components, but I need additional information. For example

• the name (i.e. key string) of intent Extras
• the destination class name of startActivity
• the route from source to sink and so on.

Is there any way to elicit those information from FlowDroid? I have read MainClass.initializeCommandLineOptions()" method in the source code (module soot-infoflow-cmd), however I can not find any corresponding parameter. Are there any materials I should read?

Best regards,

*1 

<Result>
<Sink Statement="virtualinvoke r2.<android.webkit.WebView: void loadUrl(java.lang.String)>($r4)" Method="<com.example.test_simple.Activity2: void onCreate(android.os.Bundle)>">
<AccessPath Value="$r4" Type="java.lang.String" TaintSubFields="true"/>
</Sink>
<Sources>
<Source Statement="$r2 = virtualinvoke r0.<com.example.test_simple.MainActivity: android.content.Intent getIntent()>()" Method="<com.example.test_simple.MainActivity: void onCreate(android.os.Bundle)>">
<AccessPath Value="$r2" Type="android.content.Intent" TaintSubFields="true"/>
</Source>
<Source Statement="$r3 = virtualinvoke r0.<com.example.test_simple.Activity2: android.content.Intent getIntent()>()" Method="<com.example.test_simple.Activity2: void onCreate(android.os.Bundle)>">
<AccessPath Value="$r3" Type="android.content.Intent" TaintSubFields="true"/>
</Source>
</Sources>
</Result>

[StevenArzt]

The route from source to sink is easy, you need to activate path reconstruction. In the command-line application, have a look at option -cp, which is OPTION_COMPUTE_PATHS in the code.

The destination component that receives a certain intent is not immediately accessible. FlowDroid uses it internally to augment the callgraph, but it's not visible from the API. You can try two approaches. First, you can look at the propagation path, check for a call to startActivity and see what the next line is to identify the correspondingly component. Alternatively, you can extend FlowDroid to write out such information. Have a look at the IccLink class.

The Intent extras are not relevant for Intent resolving. They do not influence which component is chosen as the receiver of the intent. Additionally, the extras often contain runtime data. Therefore, I'm not sure what you try to do here.

[yokotayokota]

Dear @StevenArzt,

I got the route from source to sink with -cp parameter. Amazing. And I see your two suggestions to get the explicit destination class name.

As you said, it is true that intent extras do not influence on flow analysis. However, when detecting security holes of benign apps, it is necessary for us to make sure that what intents injected into open components cause bad phenomena actually. I will try further to see if I can get such data from FlowDroid path information and ic3 results.

I can imagine your hard work. Thank you for your prompt response as always.

[JordanSamhi]

Hello @yokotayokota,

The easiest way to achieve what you are trying to do is to make your own app that relies on ic3 and to use the available classes (e.g., PropagationValue, FieldValue, etc.). Do not forget to use the appropriate model files so that values in extra fields are propagated correctly.

[yokotayokota]

Dear @JordanSamhi ,

Thank you so much for your quick reply. I saw your comment just as you posted it, but I could not get back to you.

If my understanding is wrong. could you please point it out?

• PropagationValue and FieldValue are classes defined in coal.
• You use these values to write out extras keys and destination class names to a model file (or database) in IC3 program.
• Your suggestion is that if I extend FlowDroid (IccLInk in Dr.Arzt's comment?) I can use PropagationValue and FieldValue in coal to get extras keys.

Do I understand you properly?

[yokotayokota]

Hello @StevenArzt ,

While waiting for improvements by Mr.Samhi, I would like to refer this tool to my colleagues. Could you tell me about your commercial code scanner that you mentioned at your comment on Sep 14? If you would like to tell me about it directory, I will send you my e-mail address. Is there any free trial of the product?

I imagine your commercial product is with good usability and well maintained to keep up with the latest environment.

[StevenArzt]

Link to the VUSC product: https://www.sit.fraunhofer.de/en/offers/projekte/vusc/

Our company (website currently in German only, but the team speaks English as well): https://secure-software.io/

[yokotayokota]

Hello @StevenArzt ,

Thank you for your product information. I will check these sites. I hope my colleagues can get a free trial service. Anyway I am still waiting for Mr.Samhi's reply. I'm looking forward to the day when I can try ICC analysis with an apk built in a recent environment.