pyre-check
pyre-check copied to clipboard
Published
20 hours ago •
facebook
facebook
Unreliable scan results when using multiple source directories in Pysa
Pysa Bug
Pre-submission checklist
- [x] I've checked the list of common issues and mine does not appear
Bug description With the same Pysa configuration, when I use multiple source directories (at least 2 or 3 source directories), sometimes the tool spots many issues in these directories, and sometimes it can't spot any issues.
Reproduction steps Suppose we are working in this folder structure:
.
├── .pyre_configuration
├── rules
│ ├── sources_sinks.pysa
│ └── taint.config
├── types
│ ├── flask
│ └── werkzeug
└── vulpy
├── bad
├── good
└── utils
-
.pyre_configuration's content:{ "taint_models_path": "rules", "search_path": "types" } -
rules/sources_sinks.pysa's content:# flask.Request sources werkzeug.wrappers.BaseRequest.path: TaintSource[UserControlled, UserControlled_Meta, URL] = ... werkzeug.wrappers.BaseRequest.full_path: TaintSource[UserControlled, UserControlled_Meta, URL] = ... werkzeug.wrappers.BaseRequest.url: TaintSource[UserControlled, UserControlled_Meta, URL] = ... werkzeug.wrappers.BaseRequest.base_url: TaintSource[UserControlled, UserControlled_Meta, URL] = ... @property def werkzeug.wrappers.BaseRequest.cookies(self) -> TaintSource[UserControlled, UserControlled_Meta, Cookies]: ... werkzeug.wrappers.BaseRequest.method: TaintSource[UserControlled, UserControlled_Meta] = ... werkzeug.wrappers.BaseRequest.headers: TaintSource[UserControlled, UserControlled_Meta, HeaderData] = ... @property def werkzeug.wrappers.CommonRequestDescriptorsMixin.content_type(self) -> TaintSource[UserControlled, UserControlled_Meta]: ... @property def werkzeug.wrappers.CommonRequestDescriptorsMixin.referrer(self) -> TaintSource[UserControlled, UserControlled_Meta, HeaderData]: ... @property def werkzeug.wrappers.UserAgentMixin.user_agent(self) -> TaintSource[UserControlled, UserControlled_Meta]: ... def werkzeug.datastructures.Headers.get(self, key, type) -> TaintSource[UserControlled, UserControlled_Meta, HeaderData]: ... flask.wrappers.Request.view_args: TaintSource[UserControlled, UserControlled_Payload] = ... werkzeug.wrappers.BaseRequest.args: TaintSource[UserControlled, UserControlled_Payload] = ... werkzeug.wrappers.BaseRequest.query_string: TaintSource[UserControlled, UserControlled_Payload] = ... @property def werkzeug.wrappers.BaseRequest.data(self) -> TaintSource[UserControlled, UserControlled_Payload]: ... werkzeug.wrappers.BaseRequest.form: TaintSource[UserControlled, UserControlled_Payload] = ... werkzeug.wrappers.BaseRequest.values: TaintSource[UserControlled, UserControlled_Payload] = ... werkzeug.wrappers.BaseRequest.files: TaintSource[UserControlled, UserControlled_Payload] = ... def werkzeug.wrappers.BaseRequest.get_data(self, cache, as_text, parse_form_data) -> TaintSource[UserControlled, UserControlled_Payload]: ... def flask.wrappers.JSONMixin.get_json(self) -> TaintSource[UserControlled, UserControlled_Payload]: ... # Sinks def sqlite3.dbapi2.Cursor.execute(self, __sql: TaintSink[SQL], __parameters): ... def sqlite3.dbapi2.Cursor.executemany(self, __sql: TaintSink[SQL], __parameters): ... def sqlite3.dbapi2.Cursor.executescript(self, __sql_script: TaintSink[SQL]): ... def sqlite3.dbapi2.Connection.execute(self, sql: TaintSink[SQL], parameters): ... def sqlite3.dbapi2.Connection.executemany(self, __sql: TaintSink[SQL], __parameters): ... def sqlite3.dbapi2.Connection.executescript(self, __sql_script: TaintSink[SQL]): ... def pathlib.PurePath.__new__(cls, *args: TaintSink[FileSystem_ReadWrite]): ... def pathlib.PurePath.__truediv__(self, key: TaintSink[FileSystem_ReadWrite]): ... def pathlib.PurePath.__rtruediv__(self, key: TaintSink[FileSystem_ReadWrite]): ... def pathlib.Path.__new__(cls, *args: TaintSink[FileSystem_ReadWrite], **kwargs): ... def pathlib.Path.glob(pattern: TaintSink[FileSystem_ReadWrite]): ... -
rules/taint.config's content:{ "sources": [ { "comment": "use to annotate all data that is controllable by the user making a request", "name": "UserControlled", "kind": "parametric" }, { "comment": "use to annotate data that is an entire payload from user requests (eg. GET/POST payloads)", "name": "UserControlled_Payload" }, { "comment": "use to annotate metadata that comes with a user's request, but is not explicitly part of the payload (headers, cookies, etc)", "name": "UserControlled_Meta" }, { "comment": "use to annotate headers, which may contain data that should be treated carefully", "name": "HeaderData" }, { "comment": "use to annotate whole or partial URLs", "name": "URL" }, { "comment": "used to annotate cookie sources", "name": "Cookies" } ], "sinks": [ { "comment": "use to annotate places of SQL injection risk", "name": "SQL" }, { "comment": "use to annotate where data flows into file system APIs that may allow read/write to an arbitrary file", "name": "FileSystem_ReadWrite" } ], "rules": [ { "name": "Possiable to SQL injection: ", "code": 5001, "sources": ["UserControlled"], "sinks": ["SQL"], "message_format": "User specified data may reach a SQL Injection" }, { "name": "Possible to path traversal: ", "code": 5002, "sources": ["UserControlled"], "sinks": ["FileSystem_ReadWrite"], "message_format": "User specified data may reach a file path traversal" } ] } -
types/flaskis Flask stub files from https://pypi.org/project/types-Flask/ -
types/werkzeugWerkzeug stub files from https://pypi.org/project/types-Werkzeug/ -
vulpyis a test repo from https://github.com/fportantier/vulpy
Run 2 commands, they are in different --source-directory options order
$ pyre --noninteractive --source-directory vulpy/good --source-directory vulpy/bad --source-directory vulpy/utils analyze
$ pyre --noninteractive --source-directory vulpy/bad --source-directory vulpy/good --source-directory vulpy/utils analyze
Expected behavior The results of the 2 above commands are the same
Logs
Please run your reproduction steps with --noninteractive (eg. pyre --noninteractive analyze) and paste the output here:
The correct result:
$ pyre --noninteractive --source-directory vulpy/bad --source-directory vulpy/good --source-directory vulpy/utils analyze
2023-04-12 15:19:06,141 [PID 533831] INFO No binary specified, looking for `pyre.bin` in PATH
2023-04-12 15:19:06,142 [PID 533831] INFO Could not determine the number of Pyre workers from configuration. Auto-set the value to 7.
2023-04-12 15:19:06,142 [PID 533831] INFO No typeshed specified, looking for it...
2023-04-12 15:19:06,142 [PID 533831] INFO Found: `/home/toor/pysa-test/lib/pyre_check/typeshed`
2023-04-12 15:19:06,143 [PID 533831] INFO Writing arguments into /tmp/pyre_arguments_alzvfqpa.json...
2023-04-12 15:19:06,144 [PID 533831] DEBUG Arguments:
{
"source_paths": {
"kind": "simple",
"paths": [
"/home/toor/pysa-test/vulpy/bad",
"/home/toor/pysa-test/vulpy/good",
"/home/toor/pysa-test/vulpy/utils"
]
},
"search_paths": [
"/home/toor/pysa-test/types",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stdlib",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/DateTimeRange",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/Deprecated",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/Flask-Cors",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/Flask-SQLAlchemy",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/JACK-Client",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/Markdown",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/Pillow",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/PyMySQL",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/PyYAML",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/Pygments",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/SQLAlchemy",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/Send2Trash",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/aiofiles",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/annoy",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/appdirs",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/atomicwrites",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/aws-xray-sdk",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/babel",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/backports.ssl_match_hostname",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/beautifulsoup4",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/bleach",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/boto",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/braintree",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/cachetools",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/caldav",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/certifi",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/chardet",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/chevron",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/click-spinner",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/colorama",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/commonmark",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/contextvars",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/croniter",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/cryptography",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/dateparser",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/decorator",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/dj-database-url",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/docopt",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/docutils",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/editdistance",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/emoji",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/entrypoints",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/first",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/flake8-2020",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/flake8-bugbear",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/flake8-builtins",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/flake8-docstrings",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/flake8-plugin-utils",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/flake8-rst-docstrings",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/flake8-simplify",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/flake8-typing-imports",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/fpdf2",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/freezegun",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/gdb",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/google-cloud-ndb",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/hdbcli",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/html5lib",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/httplib2",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/humanfriendly",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/invoke",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/jmespath",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/jsonschema",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/ldap3",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/mock",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/mypy-extensions",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/mysqlclient",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/oauthlib",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/openpyxl",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/opentracing",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/paramiko",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/parsimonious",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/passlib",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/passpy",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pep8-naming",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/playsound",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/polib",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/prettytable",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/protobuf",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/psutil",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/psycopg2",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pyOpenSSL",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pyRFC3339",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pyaudio",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pycurl",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pyfarmhash",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pyflakes",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pynput",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pysftp",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pytest-lazy-fixture",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/python-dateutil",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/python-gflags",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/python-jose",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/python-nmap",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/python-slugify",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pytz",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pyvmomi",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/redis",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/regex",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/requests",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/retry",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/selenium",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/setuptools",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/simplejson",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/singledispatch",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/six",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/slumber",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/stdlib-list",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/stripe",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/tabulate",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/termcolor",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/toml",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/toposort",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/tqdm",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/ttkthemes",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/typed-ast",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/tzlocal",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/ujson",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/urllib3",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/vobject",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/waitress",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/whatthepatch",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/xmltodict",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/xxhash",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/zxcvbn"
],
"excludes": [],
"checked_directory_allowlist": [
"/home/toor/pysa-test/vulpy/utils",
"/home/toor/pysa-test/vulpy/good",
"/home/toor/pysa-test/vulpy/bad"
],
"checked_directory_blocklist": [],
"extensions": [],
"log_path": "/home/toor/pysa-test/.pyre",
"global_root": "/home/toor/pysa-test",
"debug": false,
"python_version": {
"major": 3,
"minor": 10,
"micro": 10
},
"shared_memory": {},
"parallel": true,
"number_of_workers": 7,
"inline_decorators": false,
"no_verify": false,
"verify_dsl": false,
"strict": false,
"taint_model_paths": [
"/home/toor/pysa-test/rules"
],
"use_cache": false,
"check_invariants": false,
"limit_entrypoints": false
}
2023-04-12 15:19:06,161 [PID 533831] INFO Initializing shared memory [heap_size=8589934592, dep_table_pow=27, hash_table_pow=26]
2023-04-12 15:19:06,163 [PID 533831] INFO Verifying model syntax and configuration.
2023-04-12 15:19:06,173 [PID 533831] INFO Finding taint models in `/home/toor/pysa-test/rules`.
2023-04-12 15:19:06,173 [PID 533831] PERFORMANCE Verified model syntax and configuration: 0.000s
2023-04-12 15:19:06,173 [PID 533831] INFO Starting type checking...
2023-04-12 15:19:06,173 [PID 533831] INFO Creating environment...
2023-04-12 15:19:06,173 [PID 533831] INFO Building module tracker...
2023-04-12 15:19:06,219 [PID 533831] PERFORMANCE Module tracker built: 0.055s
2023-04-12 15:19:06,275 [PID 533831] PERFORMANCE Full environment built: 0.111s
2023-04-12 15:19:06,275 [PID 533831] INFO Collecting all definitions...
2023-04-12 15:19:07,202 [PID 533831] PERFORMANCE Collected definitions: 0.927s
2023-04-12 15:19:07,202 [PID 533831] INFO Checking 56949 functions...
2023-04-12 15:19:08,204 [PID 533831] WARNING Found untracked type `google.protobuf.struct_pb2._NullValue.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:19:08,923 [PID 533831] WARNING Found untracked type `google.protobuf.type_pb2.Field._Cardinality.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:19:08,932 [PID 533831] WARNING Found untracked type `google.protobuf.type_pb2.Field._Kind.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:19:08,942 [PID 533831] WARNING Found untracked type `google.protobuf.type_pb2._Syntax.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:19:09,201 [PID 533831] WARNING Found untracked type `google.protobuf.compiler.plugin_pb2.CodeGeneratorResponse._Feature.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:19:09,245 [PID 533831] INFO Processed 4068 of 56949 functions
2023-04-12 15:19:09,368 [PID 533831] INFO Processed 8136 of 56949 functions
2023-04-12 15:19:09,462 [PID 533831] INFO Processed 12204 of 56949 functions
2023-04-12 15:19:09,477 [PID 533831] INFO Processed 16272 of 56949 functions
2023-04-12 15:19:09,499 [PID 533831] INFO Processed 20340 of 56949 functions
2023-04-12 15:19:09,584 [PID 533831] INFO Processed 24408 of 56949 functions
2023-04-12 15:19:09,711 [PID 533831] INFO Processed 28476 of 56949 functions
2023-04-12 15:19:10,654 [PID 533831] WARNING Found untracked type `google.protobuf.descriptor_pb2.FieldDescriptorProto._Label.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:19:10,661 [PID 533831] WARNING Found untracked type `google.protobuf.descriptor_pb2.FieldDescriptorProto._Type.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:19:10,670 [PID 533831] WARNING Found untracked type `google.protobuf.descriptor_pb2.FieldOptions._CType.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:19:10,672 [PID 533831] WARNING Found untracked type `google.protobuf.descriptor_pb2.FieldOptions._JSType.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:19:10,690 [PID 533831] WARNING Found untracked type `google.protobuf.descriptor_pb2.FileOptions._OptimizeMode.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:19:10,707 [PID 533831] WARNING Found untracked type `google.protobuf.descriptor_pb2.MethodOptions._IdempotencyLevel.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:19:11,035 [PID 533831] INFO Processed 32544 of 56949 functions
2023-04-12 15:19:11,243 [PID 533831] INFO Processed 36612 of 56949 functions
2023-04-12 15:19:11,303 [PID 533831] INFO Processed 40680 of 56949 functions
2023-04-12 15:19:11,352 [PID 533831] INFO Processed 44748 of 56949 functions
2023-04-12 15:19:11,384 [PID 533831] INFO Processed 48816 of 56949 functions
2023-04-12 15:19:11,487 [PID 533831] INFO Processed 52881 of 56949 functions
2023-04-12 15:19:11,566 [PID 533831] INFO Processed 56949 of 56949 functions
2023-04-12 15:19:11,567 [PID 533831] PERFORMANCE Check_TypeCheck: 4.364s
2023-04-12 15:19:11,567 [PID 533831] MEMORY Shared memory size post-typecheck (size: 98)
2023-04-12 15:19:11,567 [PID 533831] INFO Computing class hierarchy graph...
2023-04-12 15:19:11,871 [PID 533831] PERFORMANCE Computed class hierarchy graph: 0.304s
2023-04-12 15:19:11,871 [PID 533831] INFO Computing class intervals...
2023-04-12 15:19:11,990 [PID 533831] PERFORMANCE Computed class intervals: 0.119s
2023-04-12 15:19:11,990 [PID 533831] INFO Fetching initial callables to analyze...
2023-04-12 15:19:12,417 [PID 533831] PERFORMANCE Fetched initial callables to analyze: 0.427s
2023-04-12 15:19:12,471 [PID 533831] INFO Parsing taint models...
2023-04-12 15:19:12,475 [PID 533831] INFO Finding taint models in `/home/toor/pysa-test/rules`.
2023-04-12 15:19:12,502 [PID 533831] PERFORMANCE Parsed taint models: 0.031s
2023-04-12 15:19:12,502 [PID 533831] INFO Computing inferred models...
2023-04-12 15:19:12,935 [PID 533831] PERFORMANCE Computed inferred models: 0.433s
2023-04-12 15:19:12,935 [PID 533831] INFO Computing overrides...
2023-04-12 15:19:13,254 [PID 533831] WARNING `google.protobuf.message.Message::ClearField` has 57 overrides, this might slow down the analysis considerably.
2023-04-12 15:19:13,254 [PID 533831] WARNING `google.protobuf.message.Message::__init__` has 58 overrides, this might slow down the analysis considerably.
2023-04-12 15:19:13,254 [PID 533831] WARNING `object::__eq__` has 204 overrides, this might slow down the analysis considerably.
2023-04-12 15:19:13,254 [PID 533831] WARNING `object::__hash__` has 82 overrides, this might slow down the analysis considerably.
2023-04-12 15:19:13,254 [PID 533831] WARNING `object::__init__` has 1563 overrides, this might slow down the analysis considerably.
2023-04-12 15:19:13,254 [PID 533831] WARNING `object::__ne__` has 91 overrides, this might slow down the analysis considerably.
2023-04-12 15:19:13,254 [PID 533831] WARNING `type::__call__` has 251 overrides, this might slow down the analysis considerably.
2023-04-12 15:19:13,254 [PID 533831] WARNING `type::__init__` has 1470 overrides, this might slow down the analysis considerably.
2023-04-12 15:19:13,255 [PID 533831] WARNING `type::__new__` has 176 overrides, this might slow down the analysis considerably.
2023-04-12 15:19:13,255 [PID 533831] WARNING `typing.Collection::__len__` has 57 overrides, this might slow down the analysis considerably.
2023-04-12 15:19:13,255 [PID 533831] WARNING `typing.GenericMeta::__getitem__` has 71 overrides, this might slow down the analysis considerably.
2023-04-12 15:19:13,255 [PID 533831] WARNING `typing.Iterable::__iter__` has 58 overrides, this might slow down the analysis considerably.
2023-04-12 15:19:13,255 [PID 533831] WARNING `typing.NamedTuple::__init__` has 121 overrides, this might slow down the analysis considerably.
2023-04-12 15:19:13,272 [PID 533831] PERFORMANCE Overrides computed: 0.337s
2023-04-12 15:19:13,272 [PID 533831] INFO Building call graph...
2023-04-12 15:19:13,670 [PID 533831] WARNING Found untracked type `redis.client._StrType` when comparing `redis.client.CaseInsensitiveDict` and `typing.Dict[str, typing.Any]`. The class `redis.client.CaseInsensitiveDict` will be considered a subclass of `typing.Dict[str, typing.Any]`, which could lead to false positives.
2023-04-12 15:19:14,063 [PID 533831] PERFORMANCE Call graph built: 0.791s
2023-04-12 15:19:14,063 [PID 533831] INFO Computing dependencies...
2023-04-12 15:19:14,101 [PID 533831] PERFORMANCE Computed dependencies: 0.038s
2023-04-12 15:19:14,102 [PID 533831] INFO Purging shared memory...
2023-04-12 15:19:14,109 [PID 533831] PERFORMANCE Purged shared memory: 0.008s
2023-04-12 15:19:14,109 [PID 533831] INFO Analysis fixpoint started for 14015 overrides and 498 functions...
2023-04-12 15:19:14,516 [PID 533831] PERFORMANCE Recorded initial models: 0.407s
2023-04-12 15:19:14,516 [PID 533831] INFO Iteration #0. 222 callables [...]
2023-04-12 15:19:14,581 [PID 533831] INFO Processed 222 of 222 callables
2023-04-12 15:19:14,582 [PID 533831] INFO Iteration #0, 222 callables, heap size 0.109GB took 0.07s
2023-04-12 15:19:14,582 [PID 533831] INFO Iteration #1. 165 callables [...]
2023-04-12 15:19:14,619 [PID 533831] INFO Processed 165 of 165 callables
2023-04-12 15:19:14,620 [PID 533831] INFO Iteration #1, 165 callables, heap size 0.109GB took 0.04s
2023-04-12 15:19:14,620 [PID 533831] INFO Post-processing issues for multi-source rules...
2023-04-12 15:19:14,689 [PID 533831] PERFORMANCE Finished issue post-processing for multi-source rules: 0.070s
2023-04-12 15:19:14,690 [PID 533831] INFO Found 8 issues
2023-04-12 15:19:14,690 [PID 533831] INFO Fixpoint iterations: 2
2023-04-12 15:19:14,690 [PID 533831] PERFORMANCE Analysis fixpoint complete: 0.580s
2023-04-12 15:19:14,690 [PID 533831] PERFORMANCE Analyze: 8.527s
[
{
"line": 39,
"column": 24,
"stop_line": 39,
"stop_column": 40,
"path": "vulpy/bad/mod_api.py",
"code": 5002,
"name": "Possible to path traversal",
"description": "Possible to path traversal [5002]: User specified data may reach a file path traversal",
"define": "mod_api.do_key_create"
},
{
"line": 39,
"column": 24,
"stop_line": 39,
"stop_column": 40,
"path": "vulpy/bad/mod_api.py",
"code": 5001,
"name": "Possible to SQL injection",
"description": "Possible to SQL injection [5001]: User specified data may reach a SQL Injection",
"define": "mod_api.do_key_create"
},
{
"line": 39,
"column": 42,
"stop_line": 39,
"stop_column": 58,
"path": "vulpy/bad/mod_api.py",
"code": 5001,
"name": "Possible to SQL injection",
"description": "Possible to SQL injection [5001]: User specified data may reach a SQL Injection",
"define": "mod_api.do_key_create"
},
{
"line": 80,
"column": 55,
"stop_line": 80,
"stop_column": 63,
"path": "vulpy/bad/mod_user.py",
"code": 5001,
"name": "Possible to SQL injection",
"description": "Possible to SQL injection [5001]: User specified data may reach a SQL Injection",
"define": "mod_user.do_chpasswd"
},
{
"line": 52,
"column": 23,
"stop_line": 52,
"stop_column": 31,
"path": "vulpy/bad/mod_user.py",
"code": 5001,
"name": "Possible to SQL injection",
"description": "Possible to SQL injection [5001]: User specified data may reach a SQL Injection",
"define": "mod_user.do_create"
},
{
"line": 52,
"column": 33,
"stop_line": 52,
"stop_column": 41,
"path": "vulpy/bad/mod_user.py",
"code": 5001,
"name": "Possible to SQL injection",
"description": "Possible to SQL injection [5001]: User specified data may reach a SQL Injection",
"define": "mod_user.do_create"
},
{
"line": 20,
"column": 33,
"stop_line": 20,
"stop_column": 41,
"path": "vulpy/bad/mod_user.py",
"code": 5001,
"name": "Possible to SQL injection",
"description": "Possible to SQL injection [5001]: User specified data may reach a SQL Injection",
"define": "mod_user.do_login"
},
{
"line": 20,
"column": 43,
"stop_line": 20,
"stop_column": 51,
"path": "vulpy/bad/mod_user.py",
"code": 5001,
"name": "Possible to SQL injection",
"description": "Possible to SQL injection [5001]: User specified data may reach a SQL Injection",
"define": "mod_user.do_login"
}
]
The unreliable result:
$ pyre --noninteractive --source-directory vulpy/good --source-directory vulpy/bad --source-directory vulpy/utils analyze
2023-04-12 15:21:13,437 [PID 533900] INFO No binary specified, looking for `pyre.bin` in PATH
2023-04-12 15:21:13,438 [PID 533900] INFO Could not determine the number of Pyre workers from configuration. Auto-set the value to 7.
2023-04-12 15:21:13,438 [PID 533900] INFO No typeshed specified, looking for it...
2023-04-12 15:21:13,438 [PID 533900] INFO Found: `/home/toor/pysa-test/lib/pyre_check/typeshed`
2023-04-12 15:21:13,439 [PID 533900] INFO Writing arguments into /tmp/pyre_arguments_07wr47h3.json...
2023-04-12 15:21:13,439 [PID 533900] DEBUG Arguments:
{
"source_paths": {
"kind": "simple",
"paths": [
"/home/toor/pysa-test/vulpy/good",
"/home/toor/pysa-test/vulpy/bad",
"/home/toor/pysa-test/vulpy/utils"
]
},
"search_paths": [
"/home/toor/pysa-test/types",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stdlib",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/DateTimeRange",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/Deprecated",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/Flask-Cors",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/Flask-SQLAlchemy",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/JACK-Client",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/Markdown",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/Pillow",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/PyMySQL",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/PyYAML",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/Pygments",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/SQLAlchemy",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/Send2Trash",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/aiofiles",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/annoy",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/appdirs",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/atomicwrites",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/aws-xray-sdk",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/babel",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/backports.ssl_match_hostname",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/beautifulsoup4",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/bleach",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/boto",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/braintree",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/cachetools",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/caldav",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/certifi",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/chardet",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/chevron",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/click-spinner",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/colorama",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/commonmark",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/contextvars",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/croniter",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/cryptography",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/dateparser",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/decorator",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/dj-database-url",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/docopt",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/docutils",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/editdistance",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/emoji",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/entrypoints",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/first",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/flake8-2020",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/flake8-bugbear",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/flake8-builtins",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/flake8-docstrings",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/flake8-plugin-utils",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/flake8-rst-docstrings",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/flake8-simplify",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/flake8-typing-imports",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/fpdf2",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/freezegun",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/gdb",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/google-cloud-ndb",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/hdbcli",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/html5lib",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/httplib2",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/humanfriendly",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/invoke",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/jmespath",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/jsonschema",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/ldap3",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/mock",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/mypy-extensions",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/mysqlclient",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/oauthlib",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/openpyxl",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/opentracing",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/paramiko",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/parsimonious",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/passlib",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/passpy",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pep8-naming",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/playsound",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/polib",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/prettytable",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/protobuf",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/psutil",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/psycopg2",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pyOpenSSL",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pyRFC3339",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pyaudio",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pycurl",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pyfarmhash",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pyflakes",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pynput",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pysftp",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pytest-lazy-fixture",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/python-dateutil",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/python-gflags",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/python-jose",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/python-nmap",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/python-slugify",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pytz",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/pyvmomi",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/redis",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/regex",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/requests",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/retry",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/selenium",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/setuptools",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/simplejson",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/singledispatch",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/six",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/slumber",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/stdlib-list",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/stripe",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/tabulate",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/termcolor",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/toml",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/toposort",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/tqdm",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/ttkthemes",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/typed-ast",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/tzlocal",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/ujson",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/urllib3",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/vobject",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/waitress",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/whatthepatch",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/xmltodict",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/xxhash",
"/home/toor/pysa-test/lib/pyre_check/typeshed/stubs/zxcvbn"
],
"excludes": [],
"checked_directory_allowlist": [
"/home/toor/pysa-test/vulpy/bad",
"/home/toor/pysa-test/vulpy/good",
"/home/toor/pysa-test/vulpy/utils"
],
"checked_directory_blocklist": [],
"extensions": [],
"log_path": "/home/toor/pysa-test/.pyre",
"global_root": "/home/toor/pysa-test",
"debug": false,
"python_version": {
"major": 3,
"minor": 10,
"micro": 10
},
"shared_memory": {},
"parallel": true,
"number_of_workers": 7,
"inline_decorators": false,
"no_verify": false,
"verify_dsl": false,
"strict": false,
"taint_model_paths": [
"/home/toor/pysa-test/rules"
],
"use_cache": false,
"check_invariants": false,
"limit_entrypoints": false
}
2023-04-12 15:21:13,456 [PID 533900] INFO Initializing shared memory [heap_size=8589934592, dep_table_pow=27, hash_table_pow=26]
2023-04-12 15:21:13,458 [PID 533900] INFO Verifying model syntax and configuration.
2023-04-12 15:21:13,458 [PID 533900] INFO Finding taint models in `/home/toor/pysa-test/rules`.
2023-04-12 15:21:13,458 [PID 533900] PERFORMANCE Verified model syntax and configuration: 0.000s
2023-04-12 15:21:13,459 [PID 533900] INFO Starting type checking...
2023-04-12 15:21:13,459 [PID 533900] INFO Creating environment...
2023-04-12 15:21:13,460 [PID 533900] INFO Building module tracker...
2023-04-12 15:21:13,516 [PID 533900] PERFORMANCE Module tracker built: 0.057s
2023-04-12 15:21:13,573 [PID 533900] PERFORMANCE Full environment built: 0.115s
2023-04-12 15:21:13,573 [PID 533900] INFO Collecting all definitions...
2023-04-12 15:21:14,535 [PID 533900] PERFORMANCE Collected definitions: 0.962s
2023-04-12 15:21:14,535 [PID 533900] INFO Checking 56954 functions...
2023-04-12 15:21:15,580 [PID 533900] WARNING Found untracked type `google.protobuf.struct_pb2._NullValue.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:21:16,361 [PID 533900] WARNING Found untracked type `google.protobuf.type_pb2.Field._Cardinality.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:21:16,371 [PID 533900] WARNING Found untracked type `google.protobuf.type_pb2.Field._Kind.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:21:16,382 [PID 533900] WARNING Found untracked type `google.protobuf.type_pb2._Syntax.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:21:16,612 [PID 533900] WARNING Found untracked type `google.protobuf.compiler.plugin_pb2.CodeGeneratorResponse._Feature.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:21:16,648 [PID 533900] INFO Processed 4069 of 56954 functions
2023-04-12 15:21:16,755 [PID 533900] INFO Processed 8138 of 56954 functions
2023-04-12 15:21:16,772 [PID 533900] INFO Processed 12207 of 56954 functions
2023-04-12 15:21:16,851 [PID 533900] INFO Processed 16276 of 56954 functions
2023-04-12 15:21:16,954 [PID 533900] INFO Processed 20345 of 56954 functions
2023-04-12 15:21:16,965 [PID 533900] INFO Processed 24414 of 56954 functions
2023-04-12 15:21:17,103 [PID 533900] INFO Processed 28483 of 56954 functions
2023-04-12 15:21:18,145 [PID 533900] WARNING Found untracked type `google.protobuf.descriptor_pb2.FieldDescriptorProto._Label.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:21:18,155 [PID 533900] WARNING Found untracked type `google.protobuf.descriptor_pb2.FieldDescriptorProto._Type.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:21:18,165 [PID 533900] WARNING Found untracked type `google.protobuf.descriptor_pb2.FieldOptions._CType.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:21:18,167 [PID 533900] WARNING Found untracked type `google.protobuf.descriptor_pb2.FieldOptions._JSType.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:21:18,189 [PID 533900] WARNING Found untracked type `google.protobuf.descriptor_pb2.FileOptions._OptimizeMode.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:21:18,207 [PID 533900] WARNING Found untracked type `google.protobuf.descriptor_pb2.MethodOptions._IdempotencyLevel.ValueType` when checking for attribute `DESCRIPTOR` of `google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper`.
2023-04-12 15:21:18,381 [PID 533900] INFO Processed 32552 of 56954 functions
2023-04-12 15:21:18,500 [PID 533900] INFO Processed 36621 of 56954 functions
2023-04-12 15:21:18,615 [PID 533900] INFO Processed 40690 of 56954 functions
2023-04-12 15:21:18,704 [PID 533900] INFO Processed 44759 of 56954 functions
2023-04-12 15:21:18,811 [PID 533900] INFO Processed 48816 of 56954 functions
2023-04-12 15:21:18,866 [PID 533900] INFO Processed 52885 of 56954 functions
2023-04-12 15:21:19,053 [PID 533900] INFO Processed 56954 of 56954 functions
2023-04-12 15:21:19,053 [PID 533900] PERFORMANCE Check_TypeCheck: 4.517s
2023-04-12 15:21:19,053 [PID 533900] MEMORY Shared memory size post-typecheck (size: 98)
2023-04-12 15:21:19,053 [PID 533900] INFO Computing class hierarchy graph...
2023-04-12 15:21:19,356 [PID 533900] PERFORMANCE Computed class hierarchy graph: 0.303s
2023-04-12 15:21:19,356 [PID 533900] INFO Computing class intervals...
2023-04-12 15:21:19,481 [PID 533900] PERFORMANCE Computed class intervals: 0.125s
2023-04-12 15:21:19,481 [PID 533900] INFO Fetching initial callables to analyze...
2023-04-12 15:21:19,921 [PID 533900] PERFORMANCE Fetched initial callables to analyze: 0.441s
2023-04-12 15:21:19,978 [PID 533900] INFO Parsing taint models...
2023-04-12 15:21:19,983 [PID 533900] INFO Finding taint models in `/home/toor/pysa-test/rules`.
2023-04-12 15:21:20,012 [PID 533900] PERFORMANCE Parsed taint models: 0.035s
2023-04-12 15:21:20,013 [PID 533900] INFO Computing inferred models...
2023-04-12 15:21:20,445 [PID 533900] PERFORMANCE Computed inferred models: 0.433s
2023-04-12 15:21:20,446 [PID 533900] INFO Computing overrides...
2023-04-12 15:21:20,770 [PID 533900] WARNING `google.protobuf.message.Message::ClearField` has 57 overrides, this might slow down the analysis considerably.
2023-04-12 15:21:20,770 [PID 533900] WARNING `google.protobuf.message.Message::__init__` has 58 overrides, this might slow down the analysis considerably.
2023-04-12 15:21:20,771 [PID 533900] WARNING `object::__eq__` has 204 overrides, this might slow down the analysis considerably.
2023-04-12 15:21:20,771 [PID 533900] WARNING `object::__hash__` has 82 overrides, this might slow down the analysis considerably.
2023-04-12 15:21:20,771 [PID 533900] WARNING `object::__init__` has 1563 overrides, this might slow down the analysis considerably.
2023-04-12 15:21:20,771 [PID 533900] WARNING `object::__ne__` has 91 overrides, this might slow down the analysis considerably.
2023-04-12 15:21:20,771 [PID 533900] WARNING `type::__call__` has 251 overrides, this might slow down the analysis considerably.
2023-04-12 15:21:20,771 [PID 533900] WARNING `type::__init__` has 1470 overrides, this might slow down the analysis considerably.
2023-04-12 15:21:20,772 [PID 533900] WARNING `type::__new__` has 176 overrides, this might slow down the analysis considerably.
2023-04-12 15:21:20,772 [PID 533900] WARNING `typing.Collection::__len__` has 57 overrides, this might slow down the analysis considerably.
2023-04-12 15:21:20,772 [PID 533900] WARNING `typing.GenericMeta::__getitem__` has 71 overrides, this might slow down the analysis considerably.
2023-04-12 15:21:20,772 [PID 533900] WARNING `typing.Iterable::__iter__` has 58 overrides, this might slow down the analysis considerably.
2023-04-12 15:21:20,772 [PID 533900] WARNING `typing.NamedTuple::__init__` has 121 overrides, this might slow down the analysis considerably.
2023-04-12 15:21:20,787 [PID 533900] PERFORMANCE Overrides computed: 0.342s
2023-04-12 15:21:20,788 [PID 533900] INFO Building call graph...
2023-04-12 15:21:21,577 [PID 533900] PERFORMANCE Call graph built: 0.790s
2023-04-12 15:21:21,578 [PID 533900] INFO Computing dependencies...
2023-04-12 15:21:21,615 [PID 533900] PERFORMANCE Computed dependencies: 0.038s
2023-04-12 15:21:21,616 [PID 533900] INFO Purging shared memory...
2023-04-12 15:21:21,623 [PID 533900] PERFORMANCE Purged shared memory: 0.008s
2023-04-12 15:21:21,624 [PID 533900] INFO Analysis fixpoint started for 14015 overrides and 456 functions...
2023-04-12 15:21:22,037 [PID 533900] PERFORMANCE Recorded initial models: 0.413s
2023-04-12 15:21:22,037 [PID 533900] INFO Iteration #0. 208 callables [...]
2023-04-12 15:21:22,103 [PID 533900] INFO Processed 208 of 208 callables
2023-04-12 15:21:22,103 [PID 533900] INFO Iteration #0, 208 callables, heap size 0.109GB took 0.07s
2023-04-12 15:21:22,104 [PID 533900] INFO Iteration #1. 149 callables [...]
2023-04-12 15:21:22,140 [PID 533900] INFO Processed 149 of 149 callables
2023-04-12 15:21:22,140 [PID 533900] INFO Iteration #1, 149 callables, heap size 0.109GB took 0.04s
2023-04-12 15:21:22,140 [PID 533900] INFO Post-processing issues for multi-source rules...
2023-04-12 15:21:22,210 [PID 533900] PERFORMANCE Finished issue post-processing for multi-source rules: 0.069s
2023-04-12 15:21:22,211 [PID 533900] INFO Found 0 issues
2023-04-12 15:21:22,211 [PID 533900] INFO Fixpoint iterations: 2
2023-04-12 15:21:22,211 [PID 533900] PERFORMANCE Analysis fixpoint complete: 0.587s
2023-04-12 15:21:22,211 [PID 533900] PERFORMANCE Analyze: 8.753s
[]
Additional context
- Flask stub files from https://pypi.org/project/types-Flask/
- Werkzeug stub files from https://pypi.org/project/types-Werkzeug/
Hi! It looks like you're facing a problem with inconsistent analysis results when running Pysa with different orders of source directories. This variability could be due to how Pysa manages its internal state or caching mechanisms. To get consistent results regardless of the order in which directories are specified, you might try running Pysa with a clean state each time using the --no-saved-state option. Additionally, checking for any issues in the configuration file that might affect how directories are parsed and handled could also help. If these steps don't resolve the problem, consider running Pysa with a single worker to eliminate any issues related to concurrent processing. If the issue persists, reaching out to the Pyre/Pysa community or filing a detailed bug report with the developers could be beneficial, as they might provide more specific guidance or a fix.
