Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Noisy logs due to _fetch_valid_api_metadata calls #149

Closed
acarter24 opened this issue Sep 1, 2023 · 1 comment · Fixed by #150
Closed

Noisy logs due to _fetch_valid_api_metadata calls #149

acarter24 opened this issue Sep 1, 2023 · 1 comment · Fixed by #150
Labels
Accepting Pull Requests good first issue Good for newcomers

Comments

@acarter24
Copy link
Contributor

All available dimensions and metrics are getting logged to each call, seems pretty noisy and masks what I'm looking for. I've just given an excerpt below but there's thousands of lines logged each run.

Was this intended as a debug setting rather than info level log?

tap-google-analytics/tap_google_analytics/tap.py

Line 170 in 101a165

LOGGER.info(f"**** metadata for {prop_id}") 

2023-09-01 07:54:20 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.542631Z [info     ] 2023-09-01 07:54:20,542 | INFO     | tap_google_analytics.tap | **** metadata for 403515524 cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
2023-09-01 07:54:20 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.544438Z [info     ] 2023-09-01 07:54:20,542 | INFO     | tap_google_analytics.tap | name: "properties/403515524/metadata" cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
2023-09-01 07:54:20 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.544650Z [info     ] dimensions {                   cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
2023-09-01 07:54:20 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.544818Z [info     ]   api_name: "achievementId"    cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
2023-09-01 07:54:20 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.544974Z [info     ]   ui_name: "Achievement ID"    cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
2023-09-01 07:54:20 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.545308Z [info     ]   description: "The achievement ID in a game for an event. Populated by the event parameter \'achievement_id\'." cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
2023-09-01 07:54:20 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.545472Z [info     ]   category: "Other"            cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
2023-09-01 07:54:20 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.545625Z [info     ] }                              cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
2023-09-01 07:54:20 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.545791Z [info     ] dimensions {                   cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
2023-09-01 07:54:20 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.545944Z [info     ]   api_name: "adFormat"         cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
2023-09-01 07:54:20 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.546092Z [info     ]   ui_name: "Ad format"         cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
2023-09-01 07:54:20 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.546254Z [info     ]   description: "Describes the way ads looked and where they were located. Typical formats include \'Interstitial\', \'Banner\', \'Rewarded\', and \'Native advanced\'." cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
2023-09-01 07:54:20 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.546404Z [info     ]   category: "Publisher"        cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
2023-09-01 07:54:20 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.546551Z [info     ] }                              cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
2023-09-01 07:54:20 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.546702Z [info     ] dimensions {                   cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
2023-09-01 07:54:20 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.546849Z [info     ]   api_name: "adSourceName"     cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
2023-09-01 07:54:20 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.546994Z [info     ]   ui_name: "Ad source"         cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
2023-09-01 07:54:20 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.547141Z [info     ]   description: "The source network that served the ad. Typical sources include \'AdMob Network\', \'Liftoff\', \'Facebook Audience Network\', and \'Mediated house ads\'." cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
2023-09-01 07:54:21 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.547289Z [info     ]   category: "Publisher"        cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
2023-09-01 07:54:21 +0000 - dagster - INFO - ga4_assets - c58a6f9c-c55c-44fc-ac76-5220a5e996d6 - ga4 - 2023-09-01T07:54:20.547434Z [info     ] }                              cmd_type=elb consumer=False name=tap-ga4 producer=True stdio=stderr string_id=tap-ga4
@edgarrmondragon
Copy link
Member

I think we can remove that line or change it to debug level

@acarter24 acarter24 mentioned this issue Sep 7, 2023
Merged
pnadolny13 pushed a commit that referenced this issue Sep 8, 2023
@acarter24
fix: Logging of report available dims/metrics to debug level (#150)
a4fe7db 
Closes #149
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Accepting Pull Requests good first issue Good for newcomers
Projects
MeltanoLabs Overview
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: Logging of report available dims/metrics to debug level
2 participants
@edgarrmondragon @acarter24