fix: use tomli to parse toml file, prevent decode error for new toml syntax

[PTank]

Why:

If the pyproject.toml contain some new syntax ak the dictionnary inside a list, the actual toml lib used by pip audit fail to parse the line and raise IndexError

Example who not work with toml library.

commands = [["poetry", "run", "pytest", {replace = "posargs", default = [], extend=true}]]

How:

Change with tomli and tomli_w for the dump part.

[woodruffw]

Hi @PTank, thanks for opening this. If I understand you correctly: this is a limitation in toml because it only supports v0.5 of the TOML specification, right?

Specifically, it looks like a variant of this: uiri/toml#270

Switching to tomli makes sense to me -- perhaps we could also switch to the stdlib's tomllib at some point, although that's probably a bit further down the line due to the Python version requirement + lack of writing support.

[woodruffw]

P.S.: Do you have a full pyproject.toml I could reproduce this with? That would make it easy for us to add a regression/backstop test here 🙂

[PTank]

Hi @PTank, thanks for opening this. If I understand you correctly: this is a limitation in toml because it only supports v0.5 of the TOML specification, right?
Specifically, it looks like a variant of this: uiri/toml#270

Yes, and the toml last pypi update is 2020

perhaps we could also switch to the stdlib's tomllib at some point, although that's probably a bit further down the line due to the Python version requirement + lack of writing support.

I already tested it with a switch for the < 3.11, but we do not reach 100% coverage without tools like tox for the pipeline

lines like:

import sys

if sys.version_info >= (3, 11):
    import tomllib
else:
    import tomli as tomllib

P.S.: Do you have a full pyproject.toml I could reproduce this with? That would make it easy for us to add a regression/backstop test here 🙂

[project]
dependencies = [
  "flask==2.0.1"
]
[tool.other]
must_work = ["test", {"work" = true}] # toml.decoder.TomlDecodeError: Not a homogeneous array
must_work_too = ["test", {"work" = true, other_list = []}] # IndexError: list index out of range

I did not add this to the tests, because I did not want to test toml or tomli input here

[PTank]

Hi, for the lint error, maybe a change in mypy ?
but I can fix it with a result typed dict in _cli.py

# 547
result: dict[Dependency, list[VulnerabilityResult]] = {}

[woodruffw]

Ah, don't worry about that -- I fixed that in another PR that hasn't landed yet. It's not related to these changes at all 🙂

[woodruffw]

Thanks @PTank!