sapp icon indicating copy to clipboard operation
sapp copied to clipboard

Missing Trace Frame

Open koftnt opened this issue 1 year ago • 8 comments

Hi all I got Missing Trace Frame in traces UI page very often. I want to know where can I add log to find out what happen in source code? or the codes is related to the trace frame str to show ? thanks a lot! image

koftnt avatar Oct 27 '23 03:10 koftnt

Hi @koftnt, Unfortunately, this UI is currently under maintained. I believe "Missing Trace Frame" is a problem with the tool itself producing the results (here I believe this is Mariana Trench?). This might be a bug in the implementation. Are you using sapp and mariana trench installed from pip? It is possible that there is a mismatch between those versions. You could try installing the previous version of Mariana Trench (try 1.0.3). You could also try to build sapp and mariana trench from source with the latest master.

arthaud avatar Oct 27 '23 09:10 arthaud

HI arthaud , Thank for you reply. I will try your advice. I am curious what Ui you are using internally and when will be available for public?

koftnt avatar Oct 30 '23 02:10 koftnt

We don't plan on open sourcing our internal UI because it relies on a lot of internal services.

arthaud avatar Oct 30 '23 11:10 arthaud

hi arthaud I built mariana trench from source with the latest master and sapp with the master branch. But I got error from sapp: image

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table issues has no column named first_instance_id [SQL: INSERT INTO issues (id, handle, callable_id, code, status, detected_time, first_instance_id) VALUES (?, ?, ?, ?, ?, ?, ?)] [parameters: ((3358, 'Lcom/example/myapplication/MainActivity;.onCreate:(Landroid/os/Bundle;)V:Lcom/example/myapplication/MainActivity;.launchFallback:(Landroid/content/Intent;)V:0:3:3886ec6bc93645ad', 14003, 3, 'uncategorized', 1698722093.409379, 3358), (3359, 'Lcom/example/myapplication/MainActivity;.onCreate:(Landroid/os/Bundle;)V:Lcom/example/myapplication/MainActivity$#6;.onClick:(Landroid/view/View;)V:0:1:12980548c1b721db', 14003, 1, 'uncategorized', 1698722093.409379, 3359), (3360, 'Lcom/example/myapplication/Provider;.query:(Landroid/net/Uri;[Ljava/lang/String;Ljava/lang/String;[Ljava/lang/String;Ljava/lang/String;)Landroid/database/Cursor;:Landroid/database/sqlite/SQLiteDatabase;.query:(Ljava/lang/String;[Ljava/lan:38f8ef86951646be', 14016, 4, 'uncategorized', 1698722093.409379, 3360), (3361, 'Lcom/example/myapplication/Provider;.query:(Landroid/net/Uri;[Ljava/lang/String;Ljava/lang/String;[Ljava/lang/String;Ljava/lang/String;)Landroid/database/Cursor;:Landroid/database/sqlite/SQLiteDatabase;.execSQL:(Ljava/lang/String;)V:0:4:fbe1e57d930d4601', 14016, 4, 'uncategorized', 1698722093.409379, 3361), (3362, 'Lcom/example/myapplication/WebViewActivity;.activity_lifecycle_wrapper:(Landroid/os/Bundle;)V:Lcom/example/myapplication/WebViewActivity;.onStart:()V:0:5:d39385a1fc2d3aa4', 14023, 5, 'uncategorized', 1698722093.409379, 3362), (3363, 'Lcom/example/myapplication/WebViewActivity;.activity_lifecycle_wrapper:(Landroid/os/Bundle;)V:Lcom/example/myapplication/WebViewActivity;.onStart:()V:0:2:1f9a1bf9b90c3054', 14023, 2, 'uncategorized', 1698722093.409379, 3363))] (Background on this error at: http://sqlalche.me/e/13/e3q8)

For the pip install fb-sapp case: When I run sapp --tool=mariana-trench analyze command .I got error like this: Traceback (most recent call last): File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/bin/sapp", line 8, in sys.exit(cli()) File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/click/core.py", line 1130, in call return self.main(*args, **kwargs) File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/click/decorators.py", line 84, in new_func return ctx.invoke(f, obj, *args, **kwargs) File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/cli_lib.py", line 206, in analyze pipeline.run(analysis_output, summary_blob) File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/pipeline/init.py", line 365, in run next_input, summary = step.run(next_input, summary) File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/pipeline/base_parser.py", line 230, in run self.analysis_output_to_dict_entries( File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/pipeline/base_parser.py", line 182, in analysis_output_to_dict_entries for typ, key, e in self._analysis_output_to_parsed_tuples(inputfile): File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/pipeline/base_parser.py", line 131, in _analysis_output_to_parsed_tuples for e in entries: File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/pipeline/mariana_trench_parser_v2.py", line 470, in parse yield from self.parse_handle(handle) File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/pipeline/mariana_trench_parser_v2.py", line 483, in parse_handle yield from self._parse_issues(model) File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/pipeline/mariana_trench_parser_v2.py", line 500, in _parse_issues (preconditions, final_sinks) = self._parse_issue_conditions( File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/pipeline/mariana_trench_parser_v2.py", line 561, in _parse_issue_conditions method=Method.from_json(origin), File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/pipeline/mariana_trench_parser_v2.py", line 53, in from_json canonical_name = method["name"] KeyError: 'name'

koftnt avatar Oct 31 '23 03:10 koftnt

Regarding the error:

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table issues has no column named first_instance_id

It looks like the existing schema for the issues table is out of date. The following SQL command should update it (syntax might be slightly wrong):

ALTER TABLE issues ADD COLUMN `first_instance_id` bigint unsigned DEFAULT NULL;

Alternatively, if you do not need to preserve existing data in the table, you could re-create it. Run DROP TABLE issues; then run the sapp-cli as you normally would. It should create any tables that doesn't already exist.

As for the pip-install error, it just looks like it's a really old version of the sapp-cli that won't work with the latest Mariana Trench output...

yuhshin-oss avatar Oct 31 '23 18:10 yuhshin-oss

@yuhshin-oss Regarding the error: sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table issues has no column named first_instance_id

I use sapp --tool=mariana-trench analyze command to generate sapp.db file then got the error . As a result .the db file is not generated. How can I alter table or drop table issues? Shoud I alter class Issue source code in appp and delete codes related to first_instance_id column?

I try to delete the declaration first_instance_id column code in sapp/models.py. then i got error: sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table issue_instances has no column named purge_status

koftnt avatar Nov 01 '23 07:11 koftnt

That's strange... if there isn't an existing sapp.db file, then it should create the tables with the correct schema. Can you double check that sapp.db really doesn't exist?

Alternatively, you could use the --dbname option to specify a .db file that you're sure doesn't exist:

sapp-cli --tool=mariana-trench --dbname mytmpfile.db analyze <analysis output dir>

You could also alter the sapp source code and delete relevant columns where these errors are showing up. I'd imagine that would be rather tedious though. The "first_instance_id" and "purge_status" columns are columns that we use internally and shouldn't be needed for viewing the results in the UI.

yuhshin-oss avatar Nov 01 '23 18:11 yuhshin-oss

I'm pretty sure you are using the pip fb-sapp and not fb-sapp built from master, since I see mariana_trench_parser_v2.py in the python stack trace. This won't work.

You should either use both mariana-trench and fb-sapp from pip, or build them both from master (for now, building sapp from master is broken but it will have a fix soon).

arthaud avatar Nov 11 '23 17:11 arthaud