How to handle "Failed to load report data" 504 errors (other than tweaking the nginx timeouts)?

[kastl-ars]

Dear maintainers,

I did not want to derail the nginx-focused discussion in #40, hence this separate issue.

I am getting the dreaded "Failed to load report data. Request failed with status code 504" error in our clusters when looking at something like "Last 7 days". Smaller time periods like 1 or 2 days work, but above that it is a vabanque game.

Is there any way to fix this? I already increased the timeouts in the nginx configuration (that I need to mount as a configMap in OpenShift anyway) and in the ingress / route configuration.

Ideally we would be able to get the data for the last month, but that seems out of the question currently if just a period of some days already fails.

Any ideas, tips and tricks would be highly appreciated.

Kind Regards

[lmello]

Unfortunately for large clusters we have performance issues due to opencost usage of prometheus.
My personal recommendation is to export opencost data on a daily basis using either the csv export or the https://github.com/opencost/opencost-parquet-exporter and create a table in aws athena and visualize the data in amazon quicksight or build your own visualization.

some queries for large clusters can take more than 40 minutes to run and since opencost-ui does the queries and do not cache the result of the queries it would be very unresponsive to use for large timeframes in large clusters.

[kastl-ars]

Thanks for the reply. This does not sound promising. Especially as I would not call 7 Nodes "a large cluster".

Are there any plans to implement a caching solution like kubecost uses apparently?