-
Notifications
You must be signed in to change notification settings - Fork 361
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Some sample rules from DevSkim and ApplicationInspector repo fail the verify
command.
Here is the python script I used to quickly test them all:
import os
import subprocess
import shutil
# Directory to search for JSON files
directory = r".\test"
# Locate devskim executable
devskim_path = shutil.which("devskim")
if not devskim_path:
print("Error: 'devskim' is not found in PATH. Ensure it's installed and accessible.")
exit(1)
def run_devskim_on_json_files(directory):
# Traverse directory recursively
for root, _, files in os.walk(directory):
for file in files:
if file.endswith(".json"): # Only process JSON files
file_path = os.path.join(root, file)
# print(f"Testing file: {file_path}")
try:
# Run the devskim command silently
result = subprocess.run(
[devskim_path, "verify", "-x", "Verbose", "-r", file_path],
stdout=subprocess.DEVNULL, # Suppress successful output
stderr=subprocess.DEVNULL, # Suppress error output
text=True,
shell=True
)
# If tests fail, show output and preserve colors
if result.returncode != 0:
print(f"\nTest failed for file: {file_path}")
subprocess.run(
[devskim_path, "verify", "-x", "Verbose", "-r", file_path],
shell=True # To preserve colors
)
print("\n") # Add a blank line after output
except Exception as e:
print(f"Error running devskim on {file_path}: {e}\n")
if __name__ == "__main__":
run_devskim_on_json_files(directory)
Here are the results:
Test failed for file: .\test\ai\generative_ai_libraries.json
[11:16:35 ERR] Rule AI070003 failed from unrecognized language VSProject specified
[11:16:35 ERR] Rules failed validation.
Test failed for file: .\test\cloud_services\data_storage.json
[11:16:37 ERR] Rule AI002600 failed from unrecognized language pom.xml specified
[11:16:37 ERR] Rules failed validation.
Test failed for file: .\test\cryptography\external_libraries.json
[11:16:40 ERR] Rule AI008000 failed from unrecognized language kotlin specified
[11:16:41 ERR] Rule AI008000 failed from unrecognized language scala specified
[11:16:41 ERR] Rules failed validation.
[11:16:41 ERR] Rule AI008000 failed from unrecognized language kotlin specified
[11:16:41 ERR] Rule AI008000 failed from unrecognized language scala specified
Test failed for file: .\test\data_handling\database.json
[11:16:43 ERR] Rule AI011700 failed from unrecognized language pom.xml specified
[11:16:43 ERR] Rules failed validation.
Test failed for file: .\test\device_permissions\android_intents.json
[11:16:46 ERR] Rule AI016705 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016706 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016707 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016708 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016709 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016710 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016711 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016712 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016713 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016714 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016715 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016716 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016717 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016718 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016719 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016720 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016721 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016722 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016723 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016724 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016725 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016726 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016727 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016728 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016729 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rule AI016730 failed from unrecognized language kotlin specified
[11:16:46 ERR] Rules failed validation.
[11:16:46 ERR] Rule AI016705 failed from unrecognized language kotlin specified
Test failed for file: .\test\device_permissions\UWP.json
[11:16:47 ERR] Rule AI016745 failed from unrecognized language Package.appxmanifest specified
[11:16:47 ERR] Rule AI016746 failed from unrecognized language Package.appxmanifest specified
[11:16:47 ERR] Rule AI016747 failed from unrecognized language Package.appxmanifest specified
[11:16:47 ERR] Rule AI016748 failed from unrecognized language Package.appxmanifest specified
[11:16:47 ERR] Rule AI016749 failed from unrecognized language Package.appxmanifest specified
[11:16:47 ERR] Rule AI016750 failed from unrecognized language Package.appxmanifest specified
[11:16:47 ERR] Rule AI016751 failed from unrecognized language Package.appxmanifest specified
[11:16:47 ERR] Rule AI016752 failed from unrecognized language Package.appxmanifest specified
[11:16:47 ERR] Rule AI016753 failed from unrecognized language Package.appxmanifest specified
[11:16:47 ERR] Rule AI016754 failed from unrecognized language Package.appxmanifest specified
[11:16:47 ERR] Rule AI016755 failed from unrecognized language Package.appxmanifest specified
[11:16:47 ERR] Rule AI016756 failed from unrecognized language Package.appxmanifest specified
[11:16:47 ERR] Rules failed validation.
[11:16:47 ERR] Rule AI016745 failed from unrecognized language Package.appxmanifest specified
Test failed for file: .\test\frameworks\build.json
[11:16:47 ERR] Rule AI016800 failed from unrecognized language pom.xml specified
[11:16:47 ERR] Rule AI016900 failed from unrecognized language build.xml specified
[11:16:47 ERR] Rule AI017000 failed from unrecognized language build.gradle specified
[11:16:47 ERR] Rule AI017100 failed from unrecognized language jenkins specified
[11:16:47 ERR] Rule AI017200 failed from unrecognized language sbt specified
[11:16:47 ERR] Rule AI017600 failed from unrecognized language build.xml specified
[11:16:47 ERR] Rule AI017700 failed from unrecognized language project.clj specified
[11:16:47 ERR] Rule AI017800 failed from unrecognized language VSSolution specified
[11:16:47 ERR] Rule AI017800 failed from unrecognized language VSProject specified
[11:16:47 ERR] Rules failed validation.
[11:16:47 ERR] Rule AI016800 failed from unrecognized language pom.xml specified
Test failed for file: .\test\frameworks\javascript.json
[11:16:48 ERR] Rule AI021000 failed from unrecognized language html specified
[11:16:48 ERR] Rule AI021200 failed from unrecognized language html specified
[11:16:48 ERR] Rule AI021300 failed from unrecognized language html specified
[11:16:48 ERR] Rule AI021400 failed from unrecognized language html specified
[11:16:48 ERR] Rule AI021500 failed from unrecognized language html specified
[11:16:48 ERR] Rules failed validation.
Test failed for file: .\test\general\platforms.json
[11:16:51 ERR] Rule AI028500 failed from unrecognized language VSProject specified
[11:16:51 ERR] Rule AI028600 failed from unrecognized language VSProject specified
[11:16:51 ERR] Rule AI028700 failed from unrecognized language VSProject specified
[11:16:51 ERR] Rules failed validation.
[11:16:51 ERR] Rule AI028500 failed from unrecognized language VSProject specified
Test failed for file: .\test\general\solutioninfo.json
[11:16:51 ERR] Rule AI030100 failed from unrecognized language VSsolution specified
[11:16:51 ERR] Rule AI030100 failed from unrecognized language VSProject specified
[11:16:51 ERR] Rule AI030200 failed from unrecognized language VSProject specified
[11:16:51 ERR] Rule AI030210 failed from unrecognized language VSProject specified
[11:16:51 ERR] Rule AI030220 failed from unrecognized language VSProject specified
[11:16:51 ERR] Rule AI030230 failed from unrecognized language VSProject specified
[11:16:51 ERR] Rule AI030300 failed from unrecognized language VSProject specified
[11:16:51 ERR] Rule AI030800 failed from unrecognized language pom.xml specified
[11:16:51 ERR] Rule AI030800 failed from unrecognized language build.gradle specified
[11:16:51 ERR] Rule AI030800 failed from unrecognized language build.make.xml specified
[11:16:51 ERR] Rule AI030900 failed from unrecognized language package.json specified
[11:16:51 ERR] Rule AI031000 failed from unrecognized language package.json specified
[11:16:51 ERR] Rule AI031200 failed from unrecognized language package.json specified
[11:16:51 ERR] Rule AI031400 failed from unrecognized language VSProject specified
[11:16:51 ERR] Rules failed validation.
[11:16:51 ERR] Rule AI030100 failed from unrecognized language VSsolution specified
[11:16:51 ERR] Rule AI030100 failed from unrecognized language VSProject specified
Test failed for file: .\test\infrastructure\hashicorp_terraform_tmpl.json
[11:16:52 ERR] Rule AI031566 failed from unrecognized language terraform specified
[11:16:52 ERR] Rules failed validation.
[11:16:52 ERR] Rule AI031566 failed from unrecognized language terraform specified
Test failed for file: .\test\networkcomms\outbound_network.json
[11:16:53 ERR] Rule AI032300 failed from unrecognized language html specified
[11:16:53 ERR] Rules failed validation.
Test failed for file: .\test\os\acl.json
[11:16:53 ERR] Rule AI034700 failed from unrecognized language package.json specified
[11:16:53 ERR] Rules failed validation.
Test failed for file: .\test\test_frameworks\javascript_testing.json
[11:17:09 ERR] Rule AI041800 failed from unrecognized language package.json specified
[11:17:09 ERR] Rules failed validation.
[11:17:09 ERR] Rule AI041800 failed from unrecognized language package.json specified
Test failed for file: .\test\webapp\comms.json
[11:17:11 ERR] Rule AI054500 failed from unrecognized language html specified
[11:17:11 ERR] Rule AI054600 failed from unrecognized language html specified
[11:17:11 ERR] Rule AI054700 failed from unrecognized language html specified
[11:17:11 ERR] Rules failed validation.
[11:17:11 ERR] Rule AI054500 failed from unrecognized language html specified
Test failed for file: .\test\webapp\media.json
[11:17:11 ERR] Rule AI055900 failed from unrecognized language html specified
[11:17:11 ERR] Rule AI056000 failed from unrecognized language html specified
[11:17:11 ERR] Rules failed validation.
[11:17:11 ERR] Rule AI055900 failed from unrecognized language html specified
Test failed for file: .\test\webapp\storage.json
[11:17:12 ERR] Rule AI057000 failed from unrecognized language html specified
[11:17:12 ERR] Rule AI057100 failed from unrecognized language html specified
[11:17:12 ERR] Rule AI057200 failed from unrecognized language html specified
[11:17:12 ERR] Rule AI057300 failed from unrecognized language html specified
[11:17:12 ERR] Rule AI057400 failed from unrecognized language html specified
[11:17:12 ERR] Rule AI057500 failed from unrecognized language html specified
[11:17:12 ERR] Rule AI057600 failed from unrecognized language html specified
[11:17:12 ERR] Rules failed validation.
[11:17:12 ERR] Rule AI057000 failed from unrecognized language html specified
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working