Docker overlay2 Data is too big

[yellzhi]

Summary

Synchronization starts at 0, using the snapshot data of the main network, but docker data is too big. run 7 hours later

Steps to reproduce the bug

run docker:
DOCKER_BUILDKIT=1
COMPOSE_DOCKER_CLI_BUILD=1
RESTORE_SNAPSHOT=https://update-cardano-mainnet.iohk.io/cardano-db-sync/13.6/db-sync-snapshot-schema-13.6-block-11352374-x86_64.tgz
docker-compose up -d --build &&
docker-compose logs -f

Actual Result

7 hours later ,check result:

Expected Result

docker normal operation

Environment

use https://github.com/cardano-foundation/cardano-graphql.git and --branch 8.3.3

Platform

• Linux (Ubuntu)
• Linux (Other)
• macOS
• Windows

Platform version

No response

Runtime

• Node.js
• Docker

Runtime version

No response

[Kammerlo]

Hey, are there any suspicious logs?
When restoring the data from a snapshot for db-sync, the needed storage is double than normal since it needs to store the downloaded snapshot + the data saved in the database, but once it's done the downloaded snapshot will be deleted.

If this is a bug, this will probably need to be passed to the db-sync Team, since the syncing is handled by them.

[yellzhi]

Hey, are there any suspicious logs? When restoring the data from a snapshot for db-sync, the needed storage is double than normal since it needs to store the downloaded snapshot + the data saved in the database, but once it's done the downloaded snapshot will be deleted.

If this is a bug, this will probably need to be passed to the db-sync Team, since the syncing is handled by them.

I found that the program downloading snapshots kept downloading snapshots without stopping, causing insufficient space

[yellzhi]

Hey, are there any suspicious logs? When restoring the data from a snapshot for db-sync, the needed storage is double than normal since it needs to store the downloaded snapshot + the data saved in the database, but once it's done the downloaded snapshot will be deleted.
If this is a bug, this will probably need to be passed to the db-sync Team, since the syncing is handled by them.

I found that the program downloading snapshots kept downloading snapshots without stopping, causing insufficient space

Now the way I don't use the snapshot sync block, is now synchronization is complete, but the test interface: the initialized syncPercentage and chainTipAndCurrentEpochNumber, will be always waiting for no response

[yellzhi]

Hey, are there any suspicious logs? When restoring the data from a snapshot for db-sync, the needed storage is double than normal since it needs to store the downloaded snapshot + the data saved in the database, but once it's done the downloaded snapshot will be deleted.
If this is a bug, this will probably need to be passed to the db-sync Team, since the syncing is handled by them.

I found that the program downloading snapshots kept downloading snapshots without stopping, causing insufficient space

Now the way I don't use the snapshot sync block, is now synchronization is complete, but the test interface: the initialized syncPercentage and chainTipAndCurrentEpochNumber, will be always waiting for no response

[Kammerlo]

Just for my understanding: You disabled snapshots for db-sync and started the whole stack clean again?

Could you check if there are any suspicious logs in the background, server or db-sync?

[yellzhi]

Just for my understanding: You disabled snapshots for db-sync and started the whole stack clean again?

Could you check if there are any suspicious logs in the background, server or db-sync?

yes, I started the whole stack clean again,and sync is completed, but some time use api that will wait 10 mins. and After a few queries, there will be another wait of about 10 minutes

[yellzhi]

db-sync

[Kammerlo]

Could you share logs from the server and background component?

[yellzhi]

Could you share logs from the server and background component?

this is request err:
curl -X POST -H "Content-Type: application/json" -d '{"query": "{ cardanoDbMeta { initialized syncPercentage }}"}' http://localhost:3100/graphql
{"errors":[{"message":"database query error: {"response":{"errors":[{"extensions":{"code":"unexpected","path":"$"},"message":"database query error"}],"status":200},"request":{"query":"query {\n epochs (limit: 1, order_by: { number: desc }) {\n number\n }\n cardano {\n tip {\n epoch {\n number\n }\n slotNo\n forgedAt\n }\n }}"}}","locations":[{"line":1,"column":3}],"path":["cardanoDbMeta"],"extensions":{"code":"INTERNAL_SERVER_ERROR","exception":{"response":{"errors":[{"extensions":{"code":"unexpected","path":"$"},"message":"database query error"}],"status":200},"request":{"query":"query {\n epochs (limit: 1, order_by: { number: desc }) {\n number\n }\n cardano {\n tip {\n epoch {\n number\n }\n slotNo\n forgedAt\n }\n }}"},"stacktrace":["Error: database query error: {"response":{"errors":[{"extensions":{"code":"unexpected","path":"$"},"message":"database query error"}],"status":200},"request":{"query":"query {\n epochs (limit: 1, order_by: { number: desc }) {\n number\n }\n cardano {\n tip {\n epoch {\n number\n }\n slotNo\n forgedAt\n }\n }}"}}"," at GraphQLClient. (/app/node_modules/graphql-request/dist/index.js:170:35)"," at step (/app/node_modules/graphql-request/dist/index.js:63:23)"," at Object.next (/app/node_modules/graphql-request/dist/index.js:44:53)"," at fulfilled (/app/node_modules/graphql-request/dist/index.js:35:58)"," at process.processTicksAndRejections (node:internal/process/task_queues:95:5)"]}}}],"data":null}

[yellzhi]

logs : background_1 | hasura | registry_1 | cardano-db-sync | postgres | ogmios_1 | cardano-node | cardano-graphql-server

background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"msg":"Metadata Server unreachable.","time":"2025-01-21T13:26:52.736Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"msg":"Metadata Server unreachable.","time":"2025-01-21T13:26:56.115Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"msg":"Metadata Server unreachable.","time":"2025-01-21T13:27:01.185Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"msg":"Metadata Server unreachable.","time":"2025-01-21T13:27:08.788Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"msg":"Metadata Server unreachable.","time":"2025-01-21T13:27:20.189Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"msg":"Metadata Server unreachable.","time":"2025-01-21T13:27:37.293Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"msg":"Metadata Server unreachable.","time":"2025-01-21T13:28:02.945Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"msg":"Metadata Server unreachable.","time":"2025-01-21T13:28:41.417Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":50,"msg":"connect ECONNREFUSED 172.18.0.6:80","time":"2025-01-21T13:28:41.417Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"module":"Db","msg":"Initializing...","time":"2025-01-21T13:28:50.547Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"module":"HasuraBackgroundClient","msg":"Initializing","time":"2025-01-21T13:28:50.598Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"module":"HasuraBackgroundClient","msg":"Initialized","time":"2025-01-21T13:38:31.015Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"module":"MetadataFetchClient","msg":"Initializing","time":"2025-01-21T13:38:31.015Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"msg":"Metadata Server unreachable.","time":"2025-01-21T13:38:31.023Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"msg":"Metadata Server unreachable.","time":"2025-01-21T13:38:32.027Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"msg":"Metadata Server unreachable.","time":"2025-01-21T13:38:33.531Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"msg":"Metadata Server unreachable.","time":"2025-01-21T13:38:35.785Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"msg":"Metadata Server unreachable.","time":"2025-01-21T13:38:39.164Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"msg":"Metadata Server unreachable.","time":"2025-01-21T13:38:44.233Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"msg":"Metadata Server unreachable.","time":"2025-01-21T13:38:51.831Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"msg":"Metadata Server unreachable.","time":"2025-01-21T13:39:03.233Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1aab","pid":1,"level":30,"msg":"Metadata Server unreachable.","time":"2025-01-21T13:39:20.333Z","v":0}
background_1 | {"name":"background","hostname":"f7e7c7df1

hasura_1 | {"detail":{"http_info":{"content_encoding":null,"http_version":"HTTP/1.1","ip":"172.18.0.3","method":"POST","status":200,"url":"/v1/graphql"},"operation":{"query_execution_time":6.63465e-4,"request_id":"34e50c7b-6419-45c2-9892-c17d53419a9c","request_mode":"single","request_read_time":1.848e-6,"response_size":59,"uncompressed_response_size":59,"user_vars":{"x-hasura-role":"cardano-graphql"}},"request_id":"34e50c7b-6419-45c2-9892-c17d53419a9c"},"level":"info","timestamp":"2025-01-21T13:51:23.620+0000","type":"http-log"}
hasura_1 | {"detail":{"http_info":{"content_encoding":null,"http_version":"HTTP/1.1","ip":"172.18.0.3","method":"POST","status":200,"url":"/v1/graphql"},"operation":{"query_execution_time":6.212455e-3,"request_id":"bfc9b142-4346-4981-b2bc-b3dfa74cb5e7","request_mode":"single","request_read_time":1.15e-6,"response_size":135,"uncompressed_response_size":135,"user_vars":{"x-hasura-role":"cardano-graphql"}},"request_id":"bfc9b142-4346-4981-b2bc-b3dfa74cb5e7"},"level":"info","timestamp":"2025-01-21T13:51:23.620+0000","type":"http-log"}
hasura_1 | {"detail":{"http_info":{"content_encoding":null,"http_version":"HTTP/1.1","ip":"172.18.0.3","method":"POST","status":200,"url":"/v1/graphql"},"operation":{"query_execution_time":3.525827e-3,"request_id":"440ff01e-f122-49b6-a585-36d71f9390de","request_mode":"single","request_read_time":2.018e-6,"response_size":101,"uncompressed_response_size":101,"user_vars":{"x-hasura-role":"cardano-graphql"}},"request_id":"440ff01e-f122-49b6-a585-36d71f9390de"},"level":"info","timestamp":"2025-01-21T13:51:30.569+0000","type":"http-log"}
hasura_1 | {"detail":{"http_info":{"content_encoding":null,"http_version":"HTTP/1.1","ip":"172.18.0.3","method":"POST","status":200,"url":"/v1/graphql"},"operation":{"query_execution_time":2.538735e-3,"request_id":"896dc5af-f8ed-439a-99a2-58d05b539eb6","request_mode":"single","request_read_time":1.898e-6,"response_size":101,"uncompressed_response_size":101,"user_vars":{"x-hasura-role":"cardano-graphql"}},"request_id":"896dc5af-f8ed-439a-99a2-58d05b539eb6"},"level":"info","timestamp":"2025-01-21T13:51:30.569+0000","type":"http-log"}

token-metadata-registry_1 | 2025-01-21T01:07:47.207Z INFO 11 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port 8080 (http)
token-metadata-registry_1 | 2025-01-21T01:07:47.222Z INFO 11 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
token-metadata-registry_1 | 2025-01-21T01:07:47.222Z INFO 11 --- [ main] o.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/10.1.18]
token-metadata-registry_1 | 2025-01-21T01:07:47.256Z INFO 11 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
token-metadata-registry_1 | 2025-01-21T01:07:47.257Z INFO 11 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 2613 ms
token-metadata-registry_1 | 2025-01-21T01:07:47.455Z INFO 11 --- [ main] o.apache.catalina.core.StandardContext : Suspicious URL pattern: [/**] in context [], see sections 12.1 and 12.2 of the Servlet specification
token-metadata-registry_1 | 2025-01-21T01:07:47.531Z INFO 11 --- [ main] o.c.t.r.api.config.DatasourceConfig : Trying to connect to database jdbc:postgresql://postgres:5432/cexplorer?currentSchema=tokenregistry with driver org.postgresql.Driver
token-metadata-registry_1 | 2025-01-21T01:07:47.872Z INFO 11 --- [ main] o.f.core.internal.command.DbValidate : Successfully validated 2 migrations (execution time 00:00.025s)
token-metadata-registry_1 | 2025-01-21T01:07:47.895Z INFO 11 --- [ main] o.f.core.internal.command.DbMigrate : Current version of schema "tokenregistry": 0
token-metadata-registry_1 | 2025-01-21T01:07:47.896Z INFO 11 --- [ main] o.f.core.internal.command.DbMigrate : Schema "tokenregistry" is up to date. No migration necessary.
token-metadata-registry_1 | 2025-01-21T01:07:47.965Z INFO 11 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
token-metadata-registry_1 | 2025-01-21T01:07:47.994Z INFO 11 --- [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@40c28b0d
token-metadata-registry_1 | 2025-01-21T01:07:47.996Z INFO 11 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
token-metadata-registry_1 | 2025-01-21T01:07:48.041Z INFO 11 --- [ main] o.hibernate.jpa.internal.util.LogHelper : HHH000204: Processing PersistenceUnitInfo [name: default]
token-metadata-registry_1 | 2025-01-21T01:07:48.100Z INFO 11 --- [ main] org.hibernate.Version : HHH000412: Hibernate ORM core version 6.4.1.Final
token-metadata-registry_1 | 2025-01-21T01:07:48.133Z INFO 11 --- [ main] o.h.c.internal.RegionFactoryInitiator : HHH000026: Second-level cache disabled
token-metadata-registry_1 | 2025-01-21T01:07:48.370Z INFO 11 --- [ main] o.s.o.j.p.SpringPersistenceUnitInfo : No LoadTimeWeaver setup: ignoring JPA class transformer
token-metadata-registry_1 | 2025-01-21T01:07:49.235Z INFO 11 --- [ main] o.h.e.t.j.p.i.JtaPlatformInitiator : HHH000489: No JTA platform available (set 'hibernate.transaction.jta.platform' to enable JTA platform integration)
token-metadata-registry_1 | 2025-01-21T01:07:49.238Z INFO 11 --- [ main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
token-metadata-registry_1 | 2025-01-21T01:07:49.605Z INFO 11 --- [ main] o.c.t.r.s.TokenMetadataSyncCronJob : Cronjob initialised
token-metadata-registry_1 | 2025-01-21T01:07:49.645Z WARN 11 --- [ main] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
token-metadata-registry_1 | 2025-01-21T01:07:50.104Z INFO 11 --- [ main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 2 endpoint(s) beneath base path '/actuator'
token-metadata-registry_1 | 2025-01-21T01:07:50.134Z INFO 11 --- [ main] o.s.s.web.DefaultSecurityFilterChain : Will secure any request with [org.springframework.security.web.session.DisableEncodeUrlFilter@42d4de20, org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter@2cc9a5a2, org.springframework.security.web.context.SecurityContextHolderFilter@4b0df349, org.springframework.security.web.header.HeaderWriterFilter@77732366, org.springframework.web.filter.CorsFilter@1ff35ee7, org.springframework.security.web.authentication.logout.LogoutFilter@7ed6e557, org.springframework.security.web.savedrequest.RequestCacheAwareFilter@7c070e78, org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter@639c5ab4, org.springframework.security.web.authentication.AnonymousAuthenticationFilter@4a9c188f, org.springframework.security.web.access.ExceptionTranslationFilter@470fc16c, org.springframework.security.web.access.intercept.AuthorizationFilter@5014ccf9]
token-metadata-registry_1 | 2025-01-21T01:07:50.688Z INFO 11 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port 8080 (http) with context path ''
token-metadata-registry_1 | 2025-01-21T01:07:50.706Z INFO 11 --- [ main] c.t.r.a.TokenMetadataRegistryApplication : Started TokenMetadataRegistryApplication in 6.708 seconds (process running for 7.55)
token-metadata-registry_1 | 2025-01-21T01:08:50.711Z INFO 11 --- [ scheduling-1] o.c.t.r.s.TokenMetadataSyncCronJob : about to start syncing
token-metadata-registry_1 | 2025-01-21T01:08:50.712Z INFO 11 --- [ scheduling-1] o.c.t.registry.service.GitService : exists and is git repo
token-metadata-registry_1 | 2025-01-21T04:04:29.576Z INFO 11 --- [ scheduling-1] o.c.t.r.s.TokenMetadataSyncCronJob : syncing completed
token-metadata-registry_1 | 2025-01-21T05:04:29.578Z INFO 11 --- [ scheduling-1] o.c.t.r.s.TokenMetadataSyncCronJob : about to start syncing
token-metadata-registry_1 | 2025-01-21T05:04:29.579Z INFO 11 --- [ scheduling-1] o.c.t.registry.service.GitService : exists and is git repo
token-metadata-registry_1 | 2025-01-21T07:58:45.314Z INFO 11 --- [ scheduling-1] o.c.t.r.s.TokenMetadataSyncCronJob : syncing completed
token-metadata-registry_1 | 2025-01-21T08:58:45.317Z INFO 11 --- [ scheduling-1] o.c.t.r.s.TokenMetadataSyncCronJob : about to start syncing
token-metadata-registry_1 | 2025-01-21T08:58:45.317Z INFO 11 --- [ scheduling-1] o.c.t.registry.service.GitService : exists and is git repo
token-metadata-registry_1 | 2025-01-21T11:57:20.671Z INFO 11 --- [ scheduling-1] o.c.t.r.s.TokenMetadataSyncCronJob : syncing completed
token-metadata-registry_1 | 2025-01-21T12:57:20.671Z INFO 11 --- [ scheduling-1] o.c.t.r.s.TokenMetadataSyncCronJob : about to start syncing
token-metadata-registry_1 | 2025-01-21T12:57:20.672Z INFO 11 --- [ scheduling-1] o.c.t.registry.service.GitService : exists and is git repo

cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:50:25.64 UTC] Insert Conway Block: epoch 535, slot 145901134, block 11380951, hash 52309d323135310b0c48d1f734762bc89ea2ba6837ae56c3e5d2d9d93310fcb7
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:50:25.67 UTC] Delaying delete of 1 blocks after Just 11380950 while rolling back to (slot 145901096, hash af8e8df8897abd8e936a9965b55e13c9c386a0ac8cb8bdb7d43487aee19b1783). Applying blocks until a new block is found. The node is currently at Tip (SlotNo 145901134) 90fdff769296e1e59c0676c3def7d6a6e49d384665c1430e1ae5fbc382613526 (BlockNo 11380951)
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:50:25.67 UTC] Found in memory ledger snapshot at slot 145901096, hash af8e8df8897abd8e936a9965b55e13c9c386a0ac8cb8bdb7d43487aee19b1783
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:50:25.67 UTC] Setting ConsistencyLevel to DBAheadOfLedger
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:50:25.68 UTC] Received block which is not in the db with HeaderFields {headerFieldSlot = SlotNo 145901134, headerFieldBlockNo = BlockNo 11380951, headerFieldHash = 90fdff769296e1e59c0676c3def7d6a6e49d384665c1430e1ae5fbc382613526}. Time to restore consistency.
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:50:25.68 UTC] Deleting 1 numbered equal to or greater than BlockNo 11380951
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:50:51.63 UTC]
cardano-db-sync_1 | ----------------------- Rollback Summary: -----------------------
cardano-db-sync_1 | Table: Epoch - Count: 1
cardano-db-sync_1 | Table: ReverseIndex - Count: 1
cardano-db-sync_1 | Table: Block - Count: 1
cardano-db-sync_1 | Table: TxIn - Count: 55
cardano-db-sync_1 | Table: TxOut - Count: 70
cardano-db-sync_1 | Table: MaTxOut - Count: 135
cardano-db-sync_1 | Table: CollateralTxOut - Count: 2
cardano-db-sync_1 | Table: CollateralTxIn - Count: 6
cardano-db-sync_1 | Table: ReferenceTxIn - Count: 18
cardano-db-sync_1 | Table: TxMetadata - Count: 14
cardano-db-sync_1 | Table: Withdrawal - Count: 5
cardano-db-sync_1 | Table: Redeemer - Count: 17
cardano-db-sync_1 | Table: Datum - Count: 13
cardano-db-sync_1 | Table: ExtraKeyWitness - Count: 14
cardano-db-sync_1 | Table: Tx - Count: 28
cardano-db-sync_1 |
cardano-db-sync_1 | No Deletes in tables: DrepDistr, RewardRest, PoolStat, GovActionProposal Nulled, AdaPots, EpochParam, PoolRetire, StakeRegistration, StakeDeregistration, Delegation, Treasury, Reserve, PotTransfer, MaTxMint, Script, RedeemerData, TxCbor, ParamProposal, DelegationVote, CommitteeRegistration, CommitteeDeRegistration, DrepRegistration, VotingProcedure
cardano-db-sync_1 |
cardano-db-sync_1 | Set Null: ConsumedTxOut is not active so no Nulls set
cardano-db-sync_1 |
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:50:51.63 UTC] Blocks deleted
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:50:51.71 UTC] Insert Conway Block: epoch 535, slot 145901134, block 11380951, hash 90fdff769296e1e59c0676c3def7d6a6e49d384665c1430e1ae5fbc382613526
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:50:51.72 UTC] Setting ConsistencyLevel to Consistent
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:50:51.76 UTC] Insert Conway Block: epoch 535, slot 145901135, block 11380952, hash b4043fbe4e796fe28ba6db454cc8bb689fed41dcb40edbffd5b88baf42e7461e
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:51:00.68 UTC] Insert Conway Block: epoch 535, slot 145901169, block 11380953, hash 2c7838a95a71e50edce02c3839b64f40997c5c5d5bd6aa6179fd402010fc5541
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:51:06.44 UTC] Insert Conway Block: epoch 535, slot 145901175, block 11380954, hash 6f7a94089f2e3290d17f6fd21557e7f29aa0d29fa77a3f669dc4476a5a4527e9
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:51:21.42 UTC] Insert Conway Block: epoch 535, slot 145901190, block 11380955, hash c5dc612ed192c826028378cba0581af3b79572bf861eba2a567eba85f582b16d
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:52:04.67 UTC] Insert Conway Block: epoch 535, slot 145901233, block 11380956, hash fcd7e7d3b59afc61292aa7bff0ad3cdd6c35f313f187eaceef10941c7ab71785
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:52:16.39 UTC] Insert Conway Block: epoch 535, slot 145901245, block 11380957, hash 44280a13959a06e4a1e2b5e245d18f5e9da1f78be547afe7b845fd31a2a0a4b2
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:52:27.48 UTC] Insert Conway Block: epoch 535, slot 145901256, block 11380958, hash 1b16d15674af01e1e131febbea885b532c20b0abc49e48b9606fb6652eb1a88f
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:52:54.62 UTC] Insert Conway Block: epoch 535, slot 145901283, block 11380959, hash 8c4fe4f9a9606047e084c1a6945a038fabfde68d40af3fbd6d157934f2e812f7
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:53:51.97 UTC] Insert Conway Block: continuing epoch 535 (slot 144540/432000)
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:53:51.97 UTC] Insert Conway Block: epoch 535, slot 145901340, block 11380960, hash 40e55ebb531e0239251e988414b218ec81cdd28c9b586ebf932e9e8dd6cd25ca
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:53:51.97 UTC] Pool Offchain metadata fetch: 0 results, 10 fetch errors
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:54:03.47 UTC] Insert Conway Block: epoch 535, slot 145901352, block 11380961, hash b0e0b96780faa5bc378b35875a06b1ee4d34f4298723c61dc0f4cc9bec2c86ad
cardano-db-sync_1 | [db-sync-node:Info:84] [2025-01-21 13:54:05.27 UTC] Insert Conway Block: epoch 535, slot 145901354, block 11380962, hash 7337d762490eb6217e0747654458bb0e2a208618ca12875fe4f93abb01916c6d

postgres_1 | 2025-01-21 13:50:00.464 UTC [5128] ERROR: relation "public.Cardano" does not exist at character 257
postgres_1 | 2025-01-21 13:50:00.464 UTC [5128] STATEMENT: SELECT coalesce(json_agg("root" ), '[]' ) AS "root" FROM (SELECT row_to_json((SELECT "_e" FROM (SELECT "_root.or.tip"."tip" AS "tip", "_root.or.currentEpoch"."currentEpoch" AS "currentEpoch" ) AS "_e" ) ) AS "root" FROM (SELECT * FROM "public"."Cardano" WHERE ('true') LIMIT 1 ) AS "_root.base" LEFT OUTER JOIN LATERAL (SELECT row_to_json((SELECT "_e" FROM (SELECT "_root.or.currentEpoch.base"."number" AS "number" ) AS "_e" ) ) AS "currentEpoch" FROM (SELECT * FROM "public"."Epoch" WHERE (("_root.base"."currentEpochNo") = ("number")) LIMIT 1 ) AS "_root.or.currentEpoch.base" ) AS "_root.or.currentEpoch" ON ('true') LEFT OUTER JOIN LATERAL (SELECT row_to_json((SELECT "_e" FROM (SELECT "_root.or.tip.base"."number" AS "number", ("_root.or.tip.base"."slotNo")::text AS "slotNo" ) AS "_e" ) ) AS "tip" FROM (SELECT * FROM "public"."Block" WHERE (("_root.base"."tipBlockNo") = ("number")) LIMIT 1 ) AS "_root.or.tip.base" ) AS "_root.or.tip" ON ('true') ) AS "_root"
postgres_1 | 2025-01-21 13:50:00.464 UTC [5144] ERROR: relation "public.Block" does not exist at character 464
postgres_1 | 2025-01-21 13:50:00.464 UTC [5144] STATEMENT: SELECT coalesce(json_agg("root" ORDER BY "root.pg.slotNo" DESC NULLS FIRST), '[]' ) AS "root" FROM (SELECT "_root.base"."slotNo" AS "root.pg.slotNo", row_to_json((SELECT "_e" FROM (SELECT "_root.base"."epochNo" AS "epochNo", "_root.base"."hash" AS "hash", "_root.base"."number" AS "number", ("_root.base"."slotNo")::text AS "slotNo", ("_root.base"."transactionsCount")::text AS "transactionsCount" ) AS "_e" ) ) AS "root" FROM (SELECT * FROM "public"."Block" WHERE ((("public"."Block"."number") IS NOT NULL) AND (("public"."Block"."transactionsCount") > (($2)::bigint))) ORDER BY "slotNo" DESC NULLS FIRST LIMIT 1 OFFSET 0) AS "_root.base" ORDER BY "root.pg.slotNo" DESC NULLS FIRST ) AS "_root"
postgres_1 | 2025-01-21 13:50:00.464 UTC [5141] ERROR: relation "public.Block" does not exist at character 464
postgres_1 | 2025-01-21 13:50:00.464 UTC [5141] STATEMENT: SELECT coalesce(json_agg("root" ORDER BY "root.pg.slotNo" DESC NULLS FIRST), '[]' ) AS "root" FROM (SELECT "_root.base"."slotNo" AS "root.pg.slotNo", row_to_json((SELECT "_e" FROM (SELECT "_root.base"."epochNo" AS "epochNo", "_root.base"."hash" AS "hash", "_root.base"."number" AS "number", ("_root.base"."slotNo")::text AS "slotNo", ("_root.base"."transactionsCount")::text AS "transactionsCount" ) AS "_e" ) ) AS "root" FROM (SELECT * FROM "public"."Block" WHERE ((("public"."Block"."number") IS NOT NULL) AND (("public"."Block"."transactionsCount") > (($2)::bigint))) ORDER BY "slotNo" DESC NULLS FIRST LIMIT 1 OFFSET 0) AS "_root.base" ORDER BY "root.pg.slotNo" DESC NULLS FIRST ) AS "_root"
postgres_1 | 2025-01-21 13:50:00.464 UTC [5151] ERROR: relation "public.Block" does not exist at character 464
postgres_1 | 2025-01-21 13:50:00.464 UTC [5151] STATEMENT: SELECT coalesce(json_agg("root" ORDER BY "root.pg.slotNo" DESC NULLS FIRST), '[]' ) AS "root" FROM (SELECT "_root.base"."slotNo" AS "root.pg.slotNo", row_to_json((SELECT "_e" FROM (SELECT "_root.base"."epochNo" AS "epochNo", "_root.base"."hash" AS "hash", "_root.base"."number" AS "number", ("_root.base"."slotNo")::text AS "slotNo", ("_root.base"."transactionsCount")::text AS "transactionsCount" ) AS "_e" ) ) AS "root" FROM (SELECT * FROM "public"."Block" WHERE ((("public"."Block"."number") IS NOT NULL) AND (("public"."Block"."transactionsCount") > (($2)::bigint))) ORDER BY "slotNo" DESC NULLS FIRST LIMIT 1 OFFSET 0) AS "_root.base" ORDER BY "root.pg.slotNo" DESC NULLS FIRST ) AS "_root"
postgres_1 | 2025-01-21 13:50:00.464 UTC [5155] ERROR: relation "public.Block" does not exist at character 464
postgres_1 | 2025-01-21 13:50:00.464 UTC [5155] STATEMENT: SELECT coalesce(json_agg("root" ORDER BY "root.pg.slotNo" DESC NULLS FIRST), '[]' ) AS "root" FROM (SELECT "_root.base"."slotNo" AS "root.pg.slotNo", row_to_json((SELECT "_e" FROM (SELECT "_root.base"."epochNo" AS "epochNo", "_root.base"."hash" AS "hash", "_root.base"."number" AS "number", ("_root.base"."slotNo")::text AS "slotNo", ("_root.base"."transactionsCount")::text AS "transactionsCount" ) AS "_e" ) ) AS "root" FROM (SELECT * FROM "public"."Block" WHERE ((("public"."Block"."number") IS NOT NULL) AND (("public"."Block"."transactionsCount") > (($2)::bigint))) ORDER BY "slotNo" DESC NULLS FIRST LIMIT 1 OFFSET 0) AS "_root.base" ORDER BY "root.pg.slotNo" DESC NULLS FIRST ) AS "_root"
postgres_1 | 2025-01-21 13:50:00.464 UTC [5136] ERROR: relation "public.Block" does not exist at character 464
postgres_1 | 2025-01-21 13:50:00.464 UTC [5136] STATEMENT: SELECT coalesce(json_agg("root" ORDER BY "root.pg.slotNo" DESC NULLS FIRST), '[]' ) AS "root" FROM (SELECT "_root.base"."slotNo" AS "root.pg.slotNo", row_to_json((SELECT "_e" FROM (SELECT "_root.base"."epochNo" AS "epochNo", "_root.base"."hash" AS "hash", "_root.base"."number" AS "number", ("_root.base"."slotNo")::text AS "slotNo", ("_root.base"."transactionsCount")::text AS "transactionsCount" ) AS "_e" ) ) AS "root" FROM (SELECT * FROM "public"."Block" WHERE ((("public"."Block"."number") IS NOT NULL) AND (("public"."Block"."transactionsCount") > (($2)::bigint))) ORDER BY "slotNo" DESC NULLS FIRST LIMIT 1 OFFSET 0) AS "_root.base" ORDER BY "root.pg.slotNo" DESC NULLS FIRST ) AS "_root"
postgres_1 | 2025-01-21 13:50:00.464 UTC [5118] ERROR: relation "public.Block" does not exist at character 464
postgres_1 | 2025-01-21 13:50:00.464 UTC [5118] STATEMENT: SELECT coalesce(json_agg("root" ORDER BY "root.pg.slotNo" DESC NULLS FIRST), '[]' ) AS "root" FROM (SELECT "_root.base"."slotNo" AS "root.pg.slotNo", row_to_json((SELECT "_e" FROM (SELECT "_root.base"."epochNo" AS "epochNo", "_root.base"."hash" AS "hash", "_root.base"."number" AS "number", ("_root.base"."slotNo")::text AS "slotNo", ("_root.base"."transactionsCount")::text AS "transactionsCount" ) AS "_e" ) ) AS "root" FROM (SELECT * FROM "public"."Block" WHERE ((("public"."Block"."number") IS NOT NULL) AND (("public"."Block"."transactionsCount") > (($2)::bigint))) ORDER BY "slotNo" DESC NULLS FIRST LIMIT 1 OFFSET 0) AS "_root.base" ORDER BY "root.pg.slotNo" DESC NULLS FIRST ) AS "_root"
postgres_1 | 2025-01-21 13:50:00.464 UTC [5105] ERROR: relation "public.Epoch" does not exist at character 288
postgres_1 | 2025-01-21 13:50:00.464 UTC [5105] STATEMENT: SELECT coalesce(json_agg("root" ORDER BY "root.pg.number" DESC NULLS FIRST), '[]' ) AS "root" FROM (SELECT row_to_json((SELECT "_e" FROM (SELECT "_root.base"."number" AS "number" ) AS "_e" ) ) AS "root", "_root.base"."number" AS "root.pg.number" FROM (SELECT * FROM "public"."Epoch" WHERE ('true') ORDER BY "number" DESC NULLS FIRST LIMIT 1 ) AS "_root.base" ORDER BY "root.pg.number" DESC NULLS FIRST ) AS "_root"
postgres_1 | 2025-01-21 13:50:00.465 UTC [5133] ERROR: relation "public.Block" does not exist at character 464
postgres_1 | 2025-01-21 13:50:00.465 UTC [5133] STATEMENT: SELECT coalesce(json_agg("root" ORDER BY "root.pg.slotNo" DESC NULLS FIRST), '[]' ) AS "root" FROM (SELECT "_root.base"."slotNo" AS "root.pg.slotNo", row_to_json((SELECT "_e" FROM (SELECT "_root.base"."epochNo" AS "epochNo", "_root.base"."hash" AS "hash", "_root.base"."number" AS "number", ("_root.base"."slotNo")::text AS "slotNo", ("_root.base"."transactionsCount")::text AS "transactionsCount" ) AS "_e" ) ) AS "root" FROM (SELECT * FROM "public"."Block" WHERE ((("public"."Block"."number") IS NOT NULL) AND (("public"."Block"."transactionsCount") > (($2)::bigint))) ORDER BY "slotNo" DESC NULLS FIRST LIMIT 1 OFFSET 0) AS "_root.base" ORDER BY "root.pg.slotNo" DESC NULLS FIRST ) AS "_root"
postgres_1 | 2025-01-21 13:50:00.481 UTC [5136] ERROR: relation "public.TokenMint" does not exist at character 154
postgres_1 | 2025-01-21 13:50:00.481 UTC [5136] STATEMENT: SELECT json_build_object('aggregate', json_build_object('count', COUNT(*) ) ) AS "root" FROM (SELECT 1 FROM (SELECT DISTINCT ON ("assetId") * FROM "public"."TokenMint" WHERE ('true') ) AS "_root.base" ) AS "_root"
postgres_1 | 2025-01-21 13:50:00.481 UTC [5145] ERROR: relation "public.Epoch" does not exist at character 296
postgres_1 | 2025-01-21 13:50:00.481 UTC [5145] STATEMENT: SELECT coalesce(json_agg("root" ORDER BY "root.pg.number" DESC NULLS FIRST), '[]' ) AS "root" FROM (SELECT row_to_json((SELECT "_e" FROM (SELECT "_root.or.adaPots"."adaPots" AS "adaPots" ) AS "_e" ) ) AS "root", "_root.base"."number" AS "root.pg.number" FROM (SELECT * FROM "public"."Epoch" WHERE ('true') ORDER BY "number" DESC NULLS FIRST LIMIT 1 ) AS "_root.base" LEFT OUTER JOIN LATERAL (SELECT row_to_json((SELECT "_e" FROM (SELECT ("_root.or.adaPots.base"."reserves")::text AS "reserves" ) AS "_e" ) ) AS "adaPots" FROM (SELECT * FROM "public"."ada_pots" WHERE (("_root.base"."number") = ("epoch_no")) LIMIT 1 ) AS "_root.or.adaPots.base" ) AS "_root.or.adaPots" ON ('true') ORDER BY "root.pg.number" DESC NULLS FIRST ) AS "_root"
postgres_1 | 2025-01-21 13:50:00.567 UTC [5133] ERROR: relation "public.Epoch" does not exist at character 288
postgres_1 | 2025-01-21 13:50:00.567 UTC [5133] STATEMENT: SELECT coalesce(json_agg("root" ORDER BY "root.pg.number" DESC NULLS FIRST), '[]' ) AS "root" FROM (SELECT row_to_json((SELECT "_e" FROM (SELECT "_root.base"."number" AS "number" ) AS "_e" ) ) AS "root", "_root.base"."number" AS "root.pg.number" FROM (SELECT * FROM "public"."Epoch" WHERE ('true') ORDER BY "number" DESC NULLS FIRST LIMIT 1 ) AS "_root.base" ORDER BY "root.pg.number" DESC NULLS FIRST ) AS "_root"

ogmios_1 | {"severity":"Info","timestamp":"2025-01-21T13:54:03.433536708Z","thread":"181","message":{"Health":{"tag":"HealthTick","status":{"startTime":"2025-01-21T01:07:33.465674517Z","lastKnownTip":{"slot":145901352,"id":"b0e0b96780faa5bc378b35875a06b1ee4d34f4298723c61dc0f4cc9bec2c86ad","height":11380961},"lastTipUpdate":"2025-01-21T13:54:03.433322119Z","networkSynchronization":1.00000,"currentEra":"conway","metrics":{"activeConnections":1,"runtimeStats":{"cpuTime":5028216754810,"currentHeapSize":214774,"gcCpuTime":4899884545605,"maxHeapSize":215276},"sessionDurations":{"max":0,"mean":81969,"min":0},"totalConnections":2,"totalMessages":587146,"totalUnrouted":0},"connectionStatus":"connected","currentEpoch":535,"slotInEpoch":144552,"version":"v6.10.0 (a1596d53)","network":"mainnet"}}},"version":"v6.10.0 (a1596d53)"}
ogmios_1 | {"severity":"Info","timestamp":"2025-01-21T13:54:05.277460918Z","thread":"181","message":{"Health":{"tag":"HealthTick","status":{"startTime":"2025-01-21T01:07:33.465674517Z","lastKnownTip":{"slot":145901354,"id":"7337d762490eb6217e0747654458bb0e2a208618ca12875fe4f93abb01916c6d","height":11380962},"lastTipUpdate":"2025-01-21T13:54:05.277278441Z","networkSynchronization":1.00000,"currentEra":"conway","metrics":{"activeConnections":1,"runtimeStats":{"cpuTime":5028772523010,"currentHeapSize":214763,"gcCpuTime":4900437087652,"maxHeapSize":215276},"sessionDurations":{"max":0,"mean":81969,"min":0},"totalConnections":2,"totalMessages":587146,"totalUnrouted":0},"connectionStatus":"connected","currentEpoch":535,"slotInEpoch":144554,"version":"v6.10.0 (a1596d53)","network":"mainnet"}}},"version":"v6.10.0 (a1596d53)"}
ogmios_1 | {"severity":"Info","timestamp":"2025-01-21T13:55:43.63740967Z","thread":"181","message":{"Health":{"tag":"HealthTick","status":{"startTime":"2025-01-21T01:07:33.465674517Z","lastKnownTip":{"slot":145901452,"id":"9e054fe326a80c854e5b18c5e52d99837fdf66f2d226898aec5a24416d0da012","height":11380963},"lastTipUpdate":"2025-01-21T13:55:43.637285443Z","networkSynchronization":1.00000,"currentEra":"conway","metrics":{"activeConnections":1,"runtimeStats":{"cpuTime":5049486473030,"currentHeapSize":214772,"gcCpuTime":4921075905725,"maxHeapSize":215276},"sessionDurations":{"max":0,"mean":81969,"min":0},"totalConnections":2,"totalMessages":587146,"totalUnrouted":0},"connectionStatus":"connected","currentEpoch":535,"slotInEpoch":144652,"version":"v6.10.0 (a1596d53)","network":"mainnet"}}},"version":"v6.10.0 (a1596d53)"}
ogmios_1 | {"severity":"Info","timestamp":"2025-01-21T13:55:48.085471069Z","thread":"181","message":{"Health":{"tag":"HealthTick","status":{"startTime":"2025-01-21T01:07:33.465674517Z","lastKnownTip":{"slot":145901456,"id":"e642fa4a7f2be4742aedc3b9b99cca01c3d932cd0a7484fcd2ea6ad3956a764a","height":11380964},"lastTipUpdate":"2025-01-21T13:55:48.085346437Z","networkSynchronization":1.00000,"currentEra":"conway","metrics":{"activeConnections":1,"runtimeStats":{"cpuTime":5050340281799,"currentHeapSize":214768,"gcCpuTime":4921926385939,"maxHeapSize":215276},"sessionDurations":{"max":0,"mean":81969,"min":0},"totalConnections":2,"totalMessages":587146,"totalUnrouted":0},"connectionStatus":"connected","currentEpoch":535,"slotInEpoch":144656,"version":"v6.10.0 (a1596d53)","network":"mainnet"}}},"version":"v6.10.0 (a1596d53)"}
ogmios_1 | {"severity":"Info","timestamp":"2025-01-21T13:56:34.091033306Z","thread":"181","message":{"Health":{"tag":"HealthTick","status":{"startTime":"2025-01-21T01:07:33.465674517Z","lastKnownTip":{"slot":145901502,"id":"18919ae3e7ff4d238019dcdd73de84ed095861ec5591e4c7bb5b284e4913c6d0","height":11380965},"lastTipUpdate":"2025-01-21T13:56:34.0909301Z","networkSynchronization":1.00000,"currentEra":"conway","metrics":{"activeConnections":1,"runtimeStats":{"cpuTime":5059755748419,"currentHeapSize":214776,"gcCpuTime":4931304912927,"maxHeapSize":215276},"sessionDurations":{"max":0,"mean":81969,"min":0},"totalConnections":2,"totalMessages":587146,"totalUnrouted":0},"connectionStatus":"connected","currentEpoch":535,"slotInEpoch":144702,"version":"v6.10.0 (a1596d53)","network":"mainnet"}}},"version":"v6.10.0 (a1596d53)"}

cardano-node_1 | [16b710b9:cardano.node.ChainDB:Notice:195] [2025-01-21 13:11:30.58 UTC] Chain extended, new tip: 6ae26cb9cb8b1f656bf84cf227e2f900afd0c9f1f53f9d55f84ce91b14cd3a8b at slot 145898799
cardano-node_1 | [16b710b9:cardano.node.ConnectionManager:Info:221] [2025-01-21 13:11:31.33 UTC] TrConnectionManagerCounters (ConnectionManagerCounters {fullDuplexConns = 2, duplexConns = 58, unidirectionalConns = 1, inboundConns = 2, outboundConns = 59})
cardano-node_1 | [16b710b9:cardano.node.InboundGovernor:Info:221] [2025-01-21 13:11:31.33 UTC] TrDemotedToColdRemote (ConnectionId {localAddress = 172.18.0.2:36361, remoteAddress = 66.94.103.119:6000}) (OperationSuccess KeepTr)
cardano-node_1 | [16b710b9:cardano.node.InboundGovernor:Info:221] [2025-01-21 13:11:31.33 UTC] TrInboundGovernorCounters (InboundGovernorCounters {coldPeersRemote = 56, idlePeersRemote = 0, warmPeersRemote = 1, hotPeersRemote = 1})
cardano-node_1 | [16b710b9:cardano.node.ChainDB:Notice:195] [2025-01-21 13:11:42.33 UTC] Chain extended, new tip: af5d946060115c7558792ad81ddc0e6c0a674bed319285b08c1494555d9dee03 at slot 145898811
cardano-node_1 | [16b710b9:cardano.node.ChainDB:Notice:195] [2025-01-21 13:11:45.53 UTC] Chain extended, new tip: 81ba921fa979fe2e33951d68e98a12a30992ca29dd08c5d3242175d2d49b64d9 at slot 145898814
cardano-node_1 | [16b710b9:cardano.node.ChainDB:Notice:195] [2025-01-21 13:11:47.34 UTC] Chain extended, new tip: 2f625c1d186d3ced34febd6a3cefdf4bf43676596faba8b3fa182d308a791502 at slot 145898816
cardano-node_1 | [16b710b9:cardano.node.ConnectionManager:Info:7360] [2025-01-21 13:12:07.23 UTC] TrConnectError (Just 0.0.0.0:0) 88.99.235.240:60245 Network.Socket.connect: <socket: 68>: timeout (Connection timed out)
cardano-node_1 | [16b710b9:cardano.node.ConnectionManager:Info:7360] [2025-01-21 13:12:07.23 UTC] TrConnectionManagerCounters (ConnectionManagerCounters {fullDuplexConns = 2, duplexConns = 58, unidirectionalConns = 1, inboundConns = 2, outboundConns = 59})
cardano-node_1 | [16b710b9:cardano.node.PeerSelectionCounters:Info:227] [2025-01-21 13:12:07.23 UTC] PeerSelectionView {viewRootPeers = 60, viewKnownPeers = 150, viewAvailableToConnectPeers = 149, viewColdPeersPromotions = 0, viewEstablishedPeers = 49, viewWarmPeersDemotions = 0, viewWarmPeersPromotions = 0, viewActivePeers = 20, viewActivePeersDemotions = 0, viewKnownBigLedgerPeers = 15, viewAvailableToConnectBigLedgerPeers = 14, viewColdBigLedgerPeersPromotions = 0, viewEstablishedBigLedgerPeers = 10, viewWarmBigLedgerPeersDemotions = 0, viewWarmBigLedgerPeersPromotions = 0, viewActiveBigLedgerPeers = 5, viewActiveBigLedgerPeersDemotions = 0, viewKnownLocalRootPeers = 0, viewAvailableToConnectLocalRootPeers = 0, viewColdLocalRootPeersPromotions = 0, viewEstablishedLocalRootPeers = 0, viewWarmLocalRootPeersPromotions = 0, viewActiveLocalRootPeers = 0, viewActiveLocalRootPeersDemotions = 0, viewKnownNonRootPeers = 105, viewColdNonRootPeersPromotions = 0, viewEstablishedNonRootPeers = 27, viewWarmNonRootPeersDemotions = 0, viewWarmNonRootPeersPromotions = 0, viewActiveNonRootPeers = 10, viewActiveNonRootPeersDemotions = 0, viewKnownBootstrapPeers = 0, viewColdBootstrapPeersPromotions = 0, viewEstablishedBootstrapPeers = 0, viewWarmBootstrapPeersDemotions = 0, viewWarmBootstrapPeersPromotions = 0, viewActiveBootstrapPeers = 0, viewActiveBootstrapPeersDemotions = 0}
cardano-node_1 | [16b710b9:cardano.node.PeerSelection:Info:227] [2025-01-21 13:12:07.23 UTC] TracePromoteColdFailed 50 49 88.99.235.240:60245 6.146797188923s Network.Socket.connect: <socket: 68>: timeout (Connection timed out)
cardano-node_1 | [16b710b9:cardano.node.PeerSelectionCounters:Info:227] [2025-01-21 13:12:07.23 UTC] PeerSelectionView {viewRootPeers = 60, viewKnownPeers = 150, viewAvailableToConnectPeers = 149, viewColdPeersPromotions = 1, viewEstablishedPeers = 49, viewWarmPeersDemotions = 0, viewWarmPeersPromotions = 0, viewActivePeers = 20, viewActivePeersDemotions = 0, viewKnownBigLedgerPeers = 15, viewAvailableToConnectBigLedgerPeers = 15, viewColdBigLedgerPeersPromotions = 0, viewEstablishedBigLedgerPeers = 10, viewWarmBigLedgerPeersDemotions = 0, viewWarmBigLedgerPeersPromotions = 0, viewActiveBigLedgerPeers = 5, viewActiveBigLedgerPeersDemotions = 0, viewKnownLocalRootPeers = 0, viewAvailableToConnectLocalRootPeers = 0, viewColdLocalRootPeersPromotions = 0, viewEstablishedLocalRootPeers = 0, viewWarmLocalRootPeersPromotions = 0, viewActiveLocalRootPeers = 0, viewActiveLocalRootPeersDemotions = 0, viewKnownNonRootPeers = 105, viewColdNonRootPeersPromotions = 1, viewEstablishedNonRootPeers = 27, viewWarmNonRootPeersDemotions = 0, viewWarmNonRootPeersPromotions = 0, viewActiveNonRootPeers = 10, viewActiveNonRootPeersDemotions = 0, viewKnownBootstrapPeers = 0, viewColdBootstrapPeersPromotions = 0, viewEstablishedBootstrapPeers = 0, viewWarmBootstrapPeersDemotions = 0, viewWarmBootstrapPeersPromotions = 0, viewActiveBootstrapPeers = 0, viewActiveBootstrapPeersDemotions = 0}
cardano-node_1 | [16b710b9:cardano.node.PeerSelection:Info:227] [2025-01-21 13:12:07.23 UTC] TracePromoteColdPeers 50 49 (fromList [18.198.212.156:6000])
cardano-node_1 | [16b710b9:cardano.node.ConnectionManager:Info:7390] [2025-01-21 13:12:07.23 UTC] TrConnectionManagerCounters (ConnectionManagerCounters {fullDuplexConns = 2, duplexConns = 58, unidirectionalConns = 1, inboundConns = 2, outboundConns = 59})
cardano-node_1 | [16b710b9:cardano.node.ConnectionManager:Info:7390] [2025-01-21 13:12:07.46 UTC] TrConnectionManagerCounters (ConnectionManagerCounters {fullDuplexConns = 2, duplexConns = 58, unidirectionalConns = 1, inboundConns = 2, outboundConns = 60})
cardano-node_1 | [16b710b9:cardano.node.Handshake:Info:7391] [2025-01-21 13:12:07.46 UTC] WithMuxBearer (ConnectionId {localAddress = 172.18.0.2:44195, remoteAddress = 18.198.212.156:6000}) Send (ClientAgency TokPropose,MsgProposeVersions (fromList [(NodeToNodeV_13,TList [TInt 764824073,TBool False,TInt 1,TBool False]),(NodeToNodeV_14,TList [TInt 764824073,TBool False,TInt 1,TBool False])]))

hasura_1 | {"detail":{"http_info":{"content_encoding":null,"http_version":"HTTP/1.1","ip":"172.18.0.3","method":"POST","status":200,"url":"/v1/graphql"},"operation":{"error":{"code":"unexpected","error":"database query error","internal":{"arguments":["(Oid 114,Just ("{\"x-hasura-role\":\"cardano-graphql\"}",Binary))"],"error":{"description":null,"exec_status":"FatalError","hint":null,"message":"relation "public.Epoch" does not exist","status_code":"42P01"},"prepared":true,"statement":"SELECT coalesce(json_agg("root" ORDER BY "root.pg.number" DESC NULLS FIRST), '[]' ) AS "root" FROM (SELECT row_to_json((SELECT "_e" FROM (SELECT "_root.or.adaPots"."adaPots" AS "adaPots" ) AS "_e" ) ) AS "root", "_root.base"."number" AS "root.pg.number" FROM (SELECT * FROM "public"."Epoch" WHERE ('true') ORDER BY "number" DESC NULLS FIRST LIMIT 1 ) AS "_root.base" LEFT OUTER JOIN LATERAL (SELECT row_to_json((SELECT "_e" FROM (SELECT ("_root.or.adaPots.base"."reserves")::text AS "reserves" ) AS "_e" ) ) AS "adaPots" FROM (SELECT * FROM "public"."ada_pots" WHERE (("_root.base"."number") = ("epoch_no")) LIMIT 1 ) AS "_root.or.adaPots.base" ) AS "_root.or.adaPots" ON ('true') ORDER BY "root.pg.number" DESC NULLS FIRST ) AS "_root" "},"path":"$"},"request_id":"e74fc9b7-141e-4bc0-8b8f-b4e66b40f4ce","request_mode":"error","response_size":1202,"uncompressed_response_size":1202,"user_vars":{"x-hasura-role":"cardano-graphql"}},"request_id":"e74fc9b7-141e-4bc0-8b8f-b4e66b40f4ce"},"level":"error","timestamp":"2025-01-21T13:38:25.486+0000","type":"http-log"}
hasura_1 | {"detail":{"http_info":{"content_encoding":null,"http_version":"HTTP/1.1","ip":"172.18.0.3","method":"POST","status":200,"url":"/v1/graphql"},"operation":{"query_execution_time":2.639746e-3,"request_id":"9d0e3e50-873d-4320-b784-31326fe1a92a","request_mode":"single","request_read_time":8.68e-7,"response_size":59,"uncompressed_response_size":59,"user_vars":{"x-hasura-role":"cardano-graphql"}},"request_id":"9d0e3e50-873d-4320-b784-31326fe1a92a"},"level":"info","timestamp":"2025-01-21T13:38:25.486+0000","type":"http-log"}
hasura_1 | {"detail":{"http_info":{"content_encoding":null,"http_version":"HTTP/1.1","ip":"172.18.0.3","method":"POST","status":200,"url":"/v1/graphql"},"operation":{"error":{"code":"unexpected","error":"database query error","internal":{"arguments":["(Oid 114,Just ("{\"x-hasura-role\":\"cardano-graphql\"}",Binary))"],"error":{"description":null,"exec_status":"FatalError","hint":null,"message":"relation "public.TokenMint" does not exist","status_code":"42P01"},"prepared":true,"statement":"SELECT json_build_object('aggregate', json_build_object('count', COUNT(*) ) ) AS "root" FROM (SELECT 1 FROM (SELECT DISTINCT ON ("assetId") * FROM "public"."TokenMint" WHERE ('true') ) AS "_root.base" ) AS "_root" "},"path":"$"},"request_id":"e35d4f47-6744-48bb-a815-2de1d6e67f27","request_mode":"error","response_size":593,"uncompressed_response_size":593,"user_vars":{"x-hasura-role":"cardano-graphql"}},"request_id":"e35d4f47-6744-48bb-a815-2de1d6e67f27"},"level":"error","timestamp":"2025-01-21T13:38:25.486+0000","type":"http-log"}
hasura_1 | {"detail":{"http_info":{"content_encoding":null,"http_version":"HTTP/1.1","ip":"172.18.0.3","method":"POST","status":200,"url":"/v1/graphql"},"operation":{"error":{"code":"unexpected","error":"database query error","internal":{"arguments":["(Oid 114,Just ("{\"x-hasura-role\":\"cardano-graphql\"}",Binary))"],"error":{"description":null,"exec_status":"FatalError","hint":null,"message":"relation "public.Epoch" does not exist","status_code":"42P01"},"prepared":true,"statement":"SELECT coalesce(json_agg("root" ORDER BY "root.pg.number" DESC NULLS FIRST), '[]' ) AS "root" FROM (SELECT row_to_json((SELECT "_e" FROM (SELECT "_root.base"."number" AS "number" ) AS "_e" ) ) AS "root", "_root.base"."number" AS "root.pg.number" FROM (SELECT * FROM "public"."Epoch" WHERE ('true') ORDER BY "number" DESC NULLS FIRST LIMIT 1 ) AS "_root.base" ORDER BY "root.pg.number" DESC NULLS FIRST ) AS "_root" "},"path":"$"},"request_id":"41e14959-08eb-4a8f-90b4-4ddd1bb413db","request_mode":"error","response_size":826,"uncompressed_response_size":826,"user_vars":{"x-hasura-role":"cardano-graphql"}},"request_id":"41e14959-08eb-4a8f-90b4-4ddd1bb413db"},"level":"error","timestamp":"2025-01-21T13:38:25.486+0000","type":"http-log"}
hasura_1 | {"detail":{"http_info":{"content_encoding":null,"http_version":"HTTP/1.1","ip":"172.18.0.3","method":"POST","status":200,"url":"/v1/graphql"},"operation":{"query_execution_time":1.943965e-3,"request_id":"d0d487d3-c6a3-4894-a8bd-4e248a13e1b7","request_mode":"single","request_read_time":1.632e-6,"response_size":59,"uncompressed_response_size":59,"user_vars":{"x-hasura-role":"cardano-graphql"}},"request_id":"d0d487d3-c6a3-4894-a8bd-4e248a13e1b7"},"level":"info","timestamp":"2025-01-21T13:38:30.530+0000","type":"http-log"}
hasura_1 | {"detail":{"http_info":{"content_encoding":null,"http_version":"HTTP/1.1","ip":"172.18.0.3","method":"POST","status":200,"url":"/v1/graphql"},"operation":{"query_execution_time":2.5966837e-2,"request_id":"ff54f924-767e-4374-9118-7c6c64158949","request_mode":"single","request_read_time":9.26e-7,"response_size":135,"uncompressed_response_size":135,"user_vars":{"x-hasura-role":"cardano-graphql"}},"request_id":"ff54f924-767e-4374-9118-7c6c64158949"},"level":"info","timestamp":"2025-01-21T13:38:30.530+0000","type":"http-log"}
hasura_1 | {"detail":{"http_info":{"content_encoding":null,"http_version":"HTTP/1.1","ip":"172.18.0.4","method":"POST","status":200,"url":"/v1/graphql"},"operation":{"query_execution_time":1.663733e-3,"request_id":"0156b44f-b1d2-4b99-a39a-fbc4a7dec6f6","request_mode":"single","request_read_time":2.592e-6,"response_size":36,"uncompressed_response_size":36,"user_vars":{"x-hasura-role":"cardano-graphql"}},"request_id":"0156b44f-b1d2-4b99-a39a-fbc4a7dec6f6"},"level":"info","timestamp":"2025-01-21T13:38:30.530+0000","type":"http-log"}

[yellzhi]

in logs that postgres always has errors

[Kartiiyer12]

Hi @yellzhi
When connecting to the database from hasura/background is it connecting to the the right db & schema where the db-sync synced the data in the postgres db? Looking at the errros, its looks the query is running on db which does not have the right db and schema and hence do not have these tables.

[yellzhi]

Hi @yellzhi When connecting to the database from hasura/background is it connecting to the the right db & schema where the db-sync synced the data in the postgres db? Looking at the errros, its looks the query is running on db which does not have the right db and schema and hence do not have these tables.

the cardano-graphql-background server keep restarting, maybe every 10 minutes

[Kartiiyer12]

The graphql-background, does it connect to the database and is able tp applu the schema? I can see metadata server is not reachable. Is the metadata server running, synced and the accessible? Are you doing any modifications from the default docker-compose file?