Running queries on Android application

[mzfr]

Hey,

So I am trying to learn codeql and write my own queries. I was trying to the testing of my android project but I am having some difficulties.

1. I made the database from my application by running:

codeql database create testing-app --language=java

I ran this inside the directory in which my source code is present.

2. Then in that directory, I made a new file called test-queries.ql and tried to write some code.

• I am doing this in vscode and have already installed the codeql extension for it.

Problem

When I am writing any queries errors like could not resolve module java or could not resolve module DataFlow.

I am not sure what I might be missing.

[mzfr]

I restarted by following this documentation and I've done everything that is mentioned there but now in the output section of VS code I get the following error:

[2021-01-28 14:25:50] [DETAILS] resolve library-path> Resolving query at normalized path /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java/parsing-check.ql.
[2021-01-28 14:25:50] [DETAILS] resolve library-path> Found enclosing pack 'codeql-custom-queries-java' at /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java.
[2021-01-28 14:25:50] [DETAILS] resolve library-path> Adding compilation cache at /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java/.cache.
[2021-01-28 14:25:50] [DETAILS] resolve library-path> Resolving library dependencies from /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java/qlpack.yml.
[2021-01-28 14:25:51] Exception caught at top level: Could not list directory /opt/nessus/var/nessus/audits: java.nio.file.AccessDeniedException: /opt/nessus/var/nessus/audits
                      com.semmle.frontend.packs.DiskFileSystem.directChildren(DiskFileSystem.java:98)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.pushStackEntry(PathMatchTreeWalker.java:344)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.matchDirectory(PathMatchTreeWalker.java:202)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.walk(PathMatchTreeWalker.java:123)
                      com.semmle.frontend.packs.glob.PathMatcher.walkFileTree(PathMatcher.java:263)
                      com.semmle.frontend.packs.ScanByManifests.scanFromRoot(ScanByManifests.java:412)
                      com.semmle.frontend.packs.ScanByManifests.scanUnseenInMany(ScanByManifests.java:294)
                      com.semmle.frontend.packs.ScanByManifests.scanForNamedPacks(ScanByManifests.java:227)
                      com.semmle.frontend.packs.QlPackResolver.lambda$scanForNamedPacks$1(QlPackResolver.java:97)
                      java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source)
                      com.semmle.frontend.packs.QlPackResolver.scanForNamedPacks(QlPackResolver.java:95)
                      com.semmle.frontend.packs.QLPackLibraryPathResolver.scanForPacks(QLPackLibraryPathResolver.java:107)
                      com.semmle.frontend.packs.QLPackLibraryPathResolver.resolveSinglePack(QLPackLibraryPathResolver.java:84)
                      com.semmle.cli2.resolve.ResolveLibraryPathCommand.lambda$executeJSON$1(ResolveLibraryPathCommand.java:214)
                      java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source)
                      com.semmle.cli2.resolve.ResolveLibraryPathCommand.executeJSON(ResolveLibraryPathCommand.java:231)
                      com.semmle.cli2.resolve.ResolveLibraryPathCommand.executeJSON(ResolveLibraryPathCommand.java:44)
                      com.semmle.cli2.picocli.SimpleJsonSubcommand.executeInsistingOnJSON(SimpleJsonSubcommand.java:101)
                      com.semmle.cli2.picocli.SimpleJsonSubcommand.executeSubcommand(SimpleJsonSubcommand.java:77)
                      com.semmle.cli2.picocli.SubcommandCommon.executeWithParent(SubcommandCommon.java:414)
                      com.semmle.cli2.execute.CliServerCommand.lambda$executeSubcommand$0(CliServerCommand.java:67)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:201)
                      com.semmle.cli2.execute.CliServerCommand.executeSubcommand(CliServerCommand.java:67)
                      com.semmle.cli2.picocli.SubcommandCommon.call(SubcommandCommon.java:430)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:201)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:209)
                      com.semmle.cli2.CodeQL.main(CodeQL.java:91)
A fatal error occurred: Could not list directory /opt/nessus/var/nessus/audits: java.nio.file.AccessDeniedException: /opt/nessus/var/nessus/audits

Resolving library paths failed: [2021-01-28 14:25:50] [DETAILS] resolve library-path> Resolving query at normalized path /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java/parsing-check.ql.
[2021-01-28 14:25:50] [DETAILS] resolve library-path> Found enclosing pack 'codeql-custom-queries-java' at /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java.
[2021-01-28 14:25:50] [DETAILS] resolve library-path> Adding compilation cache at /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java/.cache.
[2021-01-28 14:25:50] [DETAILS] resolve library-path> Resolving library dependencies from /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java/qlpack.yml.
[2021-01-28 14:25:51] Exception caught at top level: Could not list directory /opt/nessus/var/nessus/audits: java.nio.file.AccessDeniedException: /opt/nessus/var/nessus/audits
                      com.semmle.frontend.packs.DiskFileSystem.directChildren(DiskFileSystem.java:98)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.pushStackEntry(PathMatchTreeWalker.java:344)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.matchDirectory(PathMatchTreeWalker.java:202)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.walk(PathMatchTreeWalker.java:123)
                      com.semmle.frontend.packs.glob.PathMatcher.walkFileTree(PathMatcher.java:263)
                      com.semmle.frontend.packs.ScanByManifests.scanFromRoot(ScanByManifests.java:412)
                      com.semmle.frontend.packs.ScanByManifests.scanUnseenInMany(ScanByManifests.java:294)
                      com.semmle.frontend.packs.ScanByManifests.scanForNamedPacks(ScanByManifests.java:227)
                      com.semmle.frontend.packs.QlPackResolver.lambda$scanForNamedPacks$1(QlPackResolver.java:97)
                      java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source)
                      com.semmle.frontend.packs.QlPackResolver.scanForNamedPacks(QlPackResolver.java:95)
                      com.semmle.frontend.packs.QLPackLibraryPathResolver.scanForPacks(QLPackLibraryPathResolver.java:107)
                      com.semmle.frontend.packs.QLPackLibraryPathResolver.resolveSinglePack(QLPackLibraryPathResolver.java:84)
                      com.semmle.cli2.resolve.ResolveLibraryPathCommand.lambda$executeJSON$1(ResolveLibraryPathCommand.java:214)
                      java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source)
                      com.semmle.cli2.resolve.ResolveLibraryPathCommand.executeJSON(ResolveLibraryPathCommand.java:231)
                      com.semmle.cli2.resolve.ResolveLibraryPathCommand.executeJSON(ResolveLibraryPathCommand.java:44)
                      com.semmle.cli2.picocli.SimpleJsonSubcommand.executeInsistingOnJSON(SimpleJsonSubcommand.java:101)
                      com.semmle.cli2.picocli.SimpleJsonSubcommand.executeSubcommand(SimpleJsonSubcommand.java:77)
                      com.semmle.cli2.picocli.SubcommandCommon.executeWithParent(SubcommandCommon.java:414)
                      com.semmle.cli2.execute.CliServerCommand.lambda$executeSubcommand$0(CliServerCommand.java:67)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:201)
                      com.semmle.cli2.execute.CliServerCommand.executeSubcommand(CliServerCommand.java:67)
                      com.semmle.cli2.picocli.SubcommandCommon.call(SubcommandCommon.java:430)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:201)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:209)
                      com.semmle.cli2.CodeQL.main(CodeQL.java:91)
A fatal error occurred: Could not list directory /opt/nessus/var/nessus/audits: java.nio.file.AccessDeniedException: /opt/nessus/var/nessus/audits
 (codeQL.runQuery)
Error: Resolving library paths failed: [2021-01-28 14:25:50] [DETAILS] resolve library-path> Resolving query at normalized path /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java/parsing-check.ql.
[2021-01-28 14:25:50] [DETAILS] resolve library-path> Found enclosing pack 'codeql-custom-queries-java' at /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java.
[2021-01-28 14:25:50] [DETAILS] resolve library-path> Adding compilation cache at /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java/.cache.
[2021-01-28 14:25:50] [DETAILS] resolve library-path> Resolving library dependencies from /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java/qlpack.yml.
[2021-01-28 14:25:51] Exception caught at top level: Could not list directory /opt/nessus/var/nessus/audits: java.nio.file.AccessDeniedException: /opt/nessus/var/nessus/audits
                      com.semmle.frontend.packs.DiskFileSystem.directChildren(DiskFileSystem.java:98)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.pushStackEntry(PathMatchTreeWalker.java:344)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.matchDirectory(PathMatchTreeWalker.java:202)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.walk(PathMatchTreeWalker.java:123)
                      com.semmle.frontend.packs.glob.PathMatcher.walkFileTree(PathMatcher.java:263)
                      com.semmle.frontend.packs.ScanByManifests.scanFromRoot(ScanByManifests.java:412)
                      com.semmle.frontend.packs.ScanByManifests.scanUnseenInMany(ScanByManifests.java:294)
                      com.semmle.frontend.packs.ScanByManifests.scanForNamedPacks(ScanByManifests.java:227)
                      com.semmle.frontend.packs.QlPackResolver.lambda$scanForNamedPacks$1(QlPackResolver.java:97)
                      java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source)
                      com.semmle.frontend.packs.QlPackResolver.scanForNamedPacks(QlPackResolver.java:95)
                      com.semmle.frontend.packs.QLPackLibraryPathResolver.scanForPacks(QLPackLibraryPathResolver.java:107)
                      com.semmle.frontend.packs.QLPackLibraryPathResolver.resolveSinglePack(QLPackLibraryPathResolver.java:84)
                      com.semmle.cli2.resolve.ResolveLibraryPathCommand.lambda$executeJSON$1(ResolveLibraryPathCommand.java:214)
                      java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source)
                      com.semmle.cli2.resolve.ResolveLibraryPathCommand.executeJSON(ResolveLibraryPathCommand.java:231)
                      com.semmle.cli2.resolve.ResolveLibraryPathCommand.executeJSON(ResolveLibraryPathCommand.java:44)
                      com.semmle.cli2.picocli.SimpleJsonSubcommand.executeInsistingOnJSON(SimpleJsonSubcommand.java:101)
                      com.semmle.cli2.picocli.SimpleJsonSubcommand.executeSubcommand(SimpleJsonSubcommand.java:77)
                      com.semmle.cli2.picocli.SubcommandCommon.executeWithParent(SubcommandCommon.java:414)
                      com.semmle.cli2.execute.CliServerCommand.lambda$executeSubcommand$0(CliServerCommand.java:67)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:201)
                      com.semmle.cli2.execute.CliServerCommand.executeSubcommand(CliServerCommand.java:67)
                      com.semmle.cli2.picocli.SubcommandCommon.call(SubcommandCommon.java:430)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:201)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:209)
                      com.semmle.cli2.CodeQL.main(CodeQL.java:91)
A fatal error occurred: Could not list directory /opt/nessus/var/nessus/audits: java.nio.file.AccessDeniedException: /opt/nessus/var/nessus/audits

	at CodeQLCliServer.runCodeQlCliInternal (/home/mzfr/.vscode-insiders/extensions/github.vscode-codeql-1.3.10/out/cli.js:190:23)
	at processTicksAndRejections (internal/process/task_queues.js:94:5)
	at async CodeQLCliServer.runJsonCodeQlCliCommand (/home/mzfr/.vscode-insiders/extensions/github.vscode-codeql-1.3.10/out/cli.js:350:24)
	at async CodeQLCliServer.resolveLibraryPath (/home/mzfr/.vscode-insiders/extensions/github.vscode-codeql-1.3.10/out/cli.js:369:16)
	at async Object.compileAndRunQueryAgainstDatabase (/home/mzfr/.vscode-insiders/extensions/github.vscode-codeql-1.3.10/out/run-queries.js:370:24)
	at async compileAndRunQuery (/home/mzfr/.vscode-insiders/extensions/github.vscode-codeql-1.3.10/out/extension.js:306:26)
	at async ctx.subscriptions.push.commandRunner_1.commandRunnerWithProgress.title (/home/mzfr/.vscode-insiders/extensions/github.vscode-codeql-1.3.10/out/extension.js:338:121)

[hmakholm]

First let's see if we can reproduce the problem from the command line -- if we can, that would be easier to troubleshoot.

Can you try codeql resolve qlpacks, please? If this reproduces a failure, please provide

• the output of codeql version --format json
• the log file that codeql resolve qlpacks --logdir . will drop in the current directory

[hmakholm]

Also, do you by any chance have an ancestor directory of /opt/nessus/var/nessus/audits set up to be a VS Code workspace? I'm struggling to understand why CodeQL would even try to scan that directory.

[alexet]

Also if there is a line just above those logs that contains using CodeQL CLI: near the start followed by a command line it could be helpful to have that line.

[mzfr]

Running all those commands eventually gave me the same errors. So I deleted the directory and re-did the whole process again and now things seem to be working.

I didn't do anything differently I just followed the instruction mentioned here(https://github.com/github/vscode-codeql-starter#instructions) basically the same thing that I did before but now this is working. 🙂

[mzfr]

I did before deleting I ran: codeql resolve qlpacks --logdir . in the vscode-codeql-starter/codeql-custom-queries-cpp directory and the log file named resolve-qlpacks-20210128.175431.071.log was generated with the following content:

[2021-01-28 17:54:31] This is codeql resolve qlpacks --logdir .
[2021-01-28 17:54:31] [DETAILS] resolve qlpacks> Scanning for qlpack.yml from /opt/codeql/.codeqlmanifest.json
[2021-01-28 17:54:31] [DETAILS] resolve qlpacks> Scanning for qlpack.yml from /opt/
[2021-01-28 17:54:32] Exception caught at top level: Could not list directory /opt/nessus/var/nessus/audits: java.nio.file.AccessDeniedException: /opt/nessus/var/nessus/audits
                      com.semmle.frontend.packs.DiskFileSystem.directChildren(DiskFileSystem.java:98)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.pushStackEntry(PathMatchTreeWalker.java:344)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.matchDirectory(PathMatchTreeWalker.java:202)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.walk(PathMatchTreeWalker.java:123)
                      com.semmle.frontend.packs.glob.PathMatcher.walkFileTree(PathMatcher.java:263)
                      com.semmle.frontend.packs.ScanByManifests.scanFromRoot(ScanByManifests.java:412)
                      com.semmle.frontend.packs.ScanByManifests.scanUnseenInMany(ScanByManifests.java:294)
                      com.semmle.frontend.packs.ScanByManifests.scanForNamedPacks(ScanByManifests.java:227)
                      com.semmle.frontend.packs.QlPackResolver.lambda$scanForNamedPacks$1(QlPackResolver.java:97)
                      java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source)
                      com.semmle.frontend.packs.QlPackResolver.scanForNamedPacks(QlPackResolver.java:95)
                      com.semmle.cli2.resolve.ResolveQlpacksCommand.executeJSON(ResolveQlpacksCommand.java:41)
                      com.semmle.cli2.resolve.ResolveQlpacksCommand.executeJSON(ResolveQlpacksCommand.java:20)
                      com.semmle.cli2.picocli.SimpleJsonSubcommand.executeSubcommand(SimpleJsonSubcommand.java:80)
                      com.semmle.cli2.picocli.SubcommandCommon.call(SubcommandCommon.java:430)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:201)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:209)
                      com.semmle.cli2.CodeQL.main(CodeQL.java:91)

[mzfr]

oh btw if it's possible would you be able to look into this(github/securitylab#253)

[hmakholm]

Thanks for that log -- I can now reproduce the problem you ran into. I expect we'll be able to release a CLI version that fixes it somewhen next week.

As for the other question, I've poked our Java analysis people about it. (At least I assume it's Java -- in general it will be easier to answer questions if you state explicitly which language they're about; the code snippets in the question could also look like C# at first sight).

[mzfr]

Yes that's java. Thanks for the help
I'll mention it on that discussion

[mzfr]

Hi @hmakholm, I am sorry to tag you again but I haven't got any reply on my other question(github/securitylab#253). If it's possible can you look at it?