Can't update new CVE database

[hklkf852]

In web page show error

DataTables warning: table id=CVEs - Requested unknown parameter 'cvss4' for row 0, column 4. For more information about this error, please see http://datatables.net/tn/4

For web admin page

Database info

Collection | #records | Last update -- | -- | -- CVES | 249284 | 25-09-2024 - 05:29

CPE | 1238539 | 25-09-2024 - 05:29

CPE-other | 0 | Not updated

Capec | 559 | 24-01-2023 - 18:32

CWE | 1313 | 16-07-2024 - 08:22

via4 | 148300 | 23-01-2021 - 15:47

Whitelist: 0 rules
Blacklist: 0 rules

Database size: 6255.86MB (6.11GB)
Database size on disk: 1260.59MB (1.23GB)

[P-T-I]

Yes the schema is updated so I need to update the database that is part of the repo; hopefully have time to do it next week

[tony61282286]

thank you so much

[huganega]

I have the same problem and am waiting for the update.
Thank you so much

[specator]

You can run full repopulate... e.g. python /app/sbin/db_update.py -f from container shell. This will drop current data and update all sources with new DB schema.

[huganega]

I ran python /app/sbin/db_updater.py -f
CPE Database OK
CVE Database KeyError: 'vulnerableSystemConfidentiality'

2025-03-01 23:23:07,997 - CveXplore.main - INFO - Using mongodb as datasource, connection details: None
2025-03-01 23:23:08,004 - CveXplore.main - INFO - Failed to load Celery TaskHandler; continuing without the optional Celery backend.
2025-03-01 23:23:08,005 - CveXplore.main - INFO - Initialized CveXplore version: 0.3.36
2025-03-01 23:23:08,005 - DBUpdater - INFO - ==========================
2025-03-01 23:23:08,005 - DBUpdater - INFO - Repopulate
2025-03-01 23:23:08,005 - DBUpdater - INFO - Sat 01 March 2025 23:23
2025-03-01 23:23:08,005 - DBUpdater - INFO - ==========================
2025-03-01 23:23:08,005 - DBUpdater - INFO - Dropping metadata: cpeother
2025-03-01 23:23:08,006 - DBUpdater - INFO - Dropping metadata: mgmt_whitelist
2025-03-01 23:23:08,006 - DBUpdater - INFO - Dropping metadata: mgmt_blacklist
2025-03-01 23:23:08,006 - DBUpdater - INFO - Dropping metadata: info
2025-03-01 23:23:08,007 - DBUpdater - INFO - Dropping metadata: schema
2025-03-01 23:23:08,007 - DBUpdater - INFO - Starting initial import...
2025-03-01 23:23:08,007 - CveXplore.core.database_maintenance.main_updater - INFO - Starting Database initialization....
2025-03-01 23:23:08,013 - CveXplore.core.nvd_nist.nvd_nist_api - INFO - NVD NIST API Key found!
2025-03-01 23:23:08,013 - CveXplore.core.database_maintenance.sources_process - INFO - CPE Database population started
2025-03-01 23:23:08,014 - CveXplore.core.database_maintenance.sources_process - INFO - Starting download...
2025-03-01 23:23:08,015 - CveXplore.core.nvd_nist.nvd_nist_api - INFO - Getting count for datasource: 2
2025-03-01 23:23:08,708 - CveXplore.core.database_maintenance.sources_process - INFO - Preparing to download 1374771 CPE entries
Downloading and processing content: 0%| | 0/1374771 [00:00<?, ?it/s]2025-03-01 23:23:08,712 - CveXplore.core.nvd_nist.nvd_nist_api - INFO - Getting count for datasource: 2
Downloading and processing content: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 1374771/1374771 [04:14<00:00, 5402.04it/s]
2025-03-01 23:27:23,211 - CveXplore.core.database_maintenance.sources_process - INFO - Duration: 0:04:15.195909
2025-03-01 23:27:26,225 - CveXplore.core.database_indexer.db_indexer - INFO - Success to create index [('id', 1)] on cpe
2025-03-01 23:27:27,942 - CveXplore.core.database_indexer.db_indexer - INFO - Success to create index [('vendor', 1)] on cpe
2025-03-01 23:27:29,734 - CveXplore.core.database_indexer.db_indexer - INFO - Success to create index [('product', 1)] on cpe
2025-03-01 23:27:31,476 - CveXplore.core.database_indexer.db_indexer - INFO - Success to create index [('deprecated', 1)] on cpe
2025-03-01 23:27:33,480 - CveXplore.core.database_indexer.db_indexer - INFO - Success to create index [('cpeName', 1)] on cpe
2025-03-01 23:27:35,389 - CveXplore.core.database_indexer.db_indexer - INFO - Success to create index [('title', 1)] on cpe
2025-03-01 23:27:37,275 - CveXplore.core.database_indexer.db_indexer - INFO - Success to create index [('stem', 1)] on cpe
2025-03-01 23:27:39,237 - CveXplore.core.database_indexer.db_indexer - INFO - Success to create index [('padded_version', 1)] on cpe
2025-03-01 23:27:41,049 - CveXplore.core.database_indexer.db_indexer - INFO - Success to create index [('lastModified', 1)] on cpe
2025-03-01 23:27:41,049 - CveXplore.core.database_maintenance.sources_process - INFO - Finished CPE database population
2025-03-01 23:27:41,049 - CveXplore.core.database_maintenance.main_updater - INFO - Sleeping for 30 seconds between CPE and CVE database population..
2025-03-01 23:28:11,084 - CveXplore.core.nvd_nist.nvd_nist_api - INFO - NVD NIST API Key found!
2025-03-01 23:28:11,084 - CveXplore.core.database_maintenance.sources_process - INFO - CVE database population started
2025-03-01 23:28:11,085 - CveXplore.core.database_maintenance.sources_process - INFO - Starting CVE database population starting from year: 2000
2025-03-01 23:28:11,086 - CveXplore.core.database_maintenance.sources_process - INFO - Starting download...
2025-03-01 23:28:11,086 - CveXplore.core.nvd_nist.nvd_nist_api - INFO - Getting count for datasource: 1
2025-03-01 23:28:12,107 - CveXplore.core.database_maintenance.sources_process - INFO - Preparing to download 268787 CVE entries
Downloading and processing content: 0%| | 0/268787 [00:00<?, ?it/s]2025-03-01 23:28:12,107 - CveXplore.core.nvd_nist.nvd_nist_api - INFO - Getting count for datasource: 1
Downloading and processing content: 31%|█████████████████████████████ | 84000/268787 [06:52<15:06, 203.79it/s]
Traceback (most recent call last):
File "/app/sbin/db_updater.py", line 296, in
main(args)
File "/app/sbin/db_updater.py", line 74, in main
cvex.database.initialize()
File "/usr/local/lib/python3.10/site-packages/CveXplore/core/database_maintenance/main_updater.py", line 206, in initialize
cve_pop.populate()
File "/usr/local/lib/python3.10/site-packages/CveXplore/core/database_maintenance/sources_process.py", line 885, in populate
self.process_downloads()
File "/usr/local/lib/python3.10/site-packages/CveXplore/core/database_maintenance/sources_process.py", line 753, in process_downloads
processed_items = [
File "/usr/local/lib/python3.10/site-packages/CveXplore/core/database_maintenance/sources_process.py", line 754, in
self.process_item(item)
File "/usr/local/lib/python3.10/site-packages/CveXplore/core/database_maintenance/api_handlers.py", line 23, in process_item
item = self.process_the_item(item)
File "/usr/local/lib/python3.10/site-packages/CveXplore/core/database_maintenance/sources_process.py", line 409, in process_the_item
cve["impact4"]["vulnerable_system_confidentiality"] = item["cve"][
KeyError: 'vulnerableSystemConfidentiality'

[specator]

That's bacause NVD update source format.

In CveXplore it was fixed, but not yue published to PyPi.

So you can try to update this module directly from repo...

Or wait for new release: cve-search/cve-search#1137

[huganega]

I don't know anything about that, I have to wait

[huganega]

The following commands in the container console from cve-search (bash) fix the error:
pip install --force-reinstall cvexplore==0.3.38
pip install --upgrade pip
/app/sbin/db_updater.py -f