How to extract all the key value pairs from properties files

[sbilello]

Hello!
Let's say we a spring java boot application with several properties file for internationalization.

src/resources/language_it.properties 
src/resources/language_en.properties 

Is there a way to export all the property values and their location via codeql or is it better to do it via bash script?

[aeisenberg]

Sure, you can do this in codeql. Something like this will grab all key value pairs from something.properties. However, depending on what your use case is, it might be simpler just to use bash.

import java
import semmle.code.configfiles.ConfigFiles

from ConfigPair cp
where cp.getFile().getBaseName() = "something.properties"
select cp.getLocation(), cp.getNameElement(), cp.getValueElement()

[sbilello]

Thank you very much!
I am adding for anyone else an important detail:
I rebuilt the database but I had to set this variable before doing it
export LGTM_INDEX_PROPERTIES_FILES=true

❯ cat /usr/local/opt/codeql/java/tools/pre-finalize.sh
#!/bin/bash

set -eu
if [ "${LGTM_INDEX_XML_MODE:-default}" == "default" ]; then
"$CODEQL_DIST/codeql" database index-files \
    --include "**/AndroidManifest.xml" \
    --include "**/pom.xml" \
    --include "**/struts.xml" \
    --include "**/web.xml" \
    --size-limit 10m \
    --language xml \
    -- \
    "$CODEQL_EXTRACTOR_JAVA_WIP_DATABASE"
elif [ "${LGTM_INDEX_XML_MODE}" == "smart" ]; then
export CODEQL_EXTRACTOR_XML_PRIMARY_TAGS="faceted-project project plugin idea-plugin beans struts web-app module ui:UiBinder persistence"
"$CODEQL_DIST/codeql" database index-files \
    --include-extension=.xml \
    --size-limit 10m \
    --language xml \
    -- \
    "$CODEQL_EXTRACTOR_JAVA_WIP_DATABASE"
elif [ "${LGTM_INDEX_XML_MODE}" == "all" ]; then
"$CODEQL_DIST/codeql" database index-files \
    --include-extension=.xml \
    --size-limit 10m \
    --language xml \
    -- \
    "$CODEQL_EXTRACTOR_JAVA_WIP_DATABASE"
fi

if [ "${LGTM_INDEX_PROPERTIES_FILES:-false}" == "true" ]; then
"$CODEQL_DIST/codeql" database index-files \
    --include-extension=.properties \
    --size-limit=5m \
    --language properties \
    -- \
    "$CODEQL_EXTRACTOR_JAVA_WIP_DATABASE"
fi
❯ pwd

I don't see in the script the YAML extension. Is not it supported? @aeisenberg

[aeisenberg]

Thanks for pointing out the extra environment variable. Forgot to mention that.

YAML support exists, but it is part of the JavaScript language libraries. If you want to query yaml files, you will need to create a second database for your project, using the javascript extractor.

[luchua-bc]

That is great the software can handle properties. Could the YAML support be ported to the Java language libraries as well since YAML is also pretty popular with Java?

[aeisenberg]

At this point, since there is a workaround (create a javascript database), this would be low priority for us. However, please create a feature request and we can look into this in the future.