Skip to content

Regarding Google Cloud Data Storage, can you talk about the specific configuration steps? #2140

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bobzhou80 opened this issue Mar 21, 2019 · 1 comment

Comments

@bobzhou80
Copy link

You will also need to enable Datastore support for your new project using the following link:
https://console.cloud.google.com/datastore/setup?project=

Regarding Google Cloud Data Storage, can you talk about the specific configuration steps? I am running locally without docker, I have encountered the following problem, I don't know if there is a problem with the storage settings. First time using cloud storage

zb:app zb$ GAE_SERVICE="analyzer" dart bin/server.dart
2019-03-21 15:35:53.438945 Warning: rediscache: using in-memory cache instead of redis
2019-03-21 15:35:53.446235 Info: pub.analyzer: About to start frontend isolate #1...
2019-03-21 15:35:53.562407 Info: pub.analyzer: Frontend isolate #1 started.
2019-03-21 15:35:53.563381 Warning: pub.analyzer: FLUTTER_SDK is not set, assuming flutter is in PATH.
2019-03-21 15:35:53.564427 Info: pub.analyzer: About to start worker isolate #1...
2019-03-21 15:35:53.695322 Info: pub.analyzer: Worker isolate #1 started.
2019-03-21 15:35:54.132673 Warning: rediscache: using in-memory cache instead of redis
2019-03-21 15:35:54.241747 Warning: rediscache: using in-memory cache instead of redis
2019-03-21 15:35:54.627339 Info: pub.popularity: Loading popularity data (init): gs://dart-app-235113--popularity/v3/popularity.json.gz
2019-03-21 15:35:54.715277 Info: pub.popularity: Loading popularity data (init): gs://dart-app-235113--popularity/v3/popularity.json.gz
2019-03-21 15:35:54.895976 Error: pub.popularity: Unable to load popularity data: gs://dart-app-235113--popularity/v3/popularity.json.gz

Error:
DetailedApiRequestError(status: 404, message: No error details. HTTP status was: 404.)

Stack:
#0 _validateResponse (package:_discoveryapis_commons/src/clients.dart:879:5)

#1 ApiRequester.request (package:_discoveryapis_commons/src/clients.dart:69:22)

#2 ObjectsResourceApi.get (package:googleapis/storage/v1.dart:2832:32)
#3 _BucketImpl.read (file:///Users/zb/.pub-cache/hosted/pub.dartlang.org/gcloud-0.6.0+2/lib/src/storage_impl.dart:224:10)

#4 PopularityStorage.fetch (package:pub_dartlang_org/shared/popularity_storage.dart:62:12)

#5 PopularityStorage.init (package:pub_dartlang_org/shared/popularity_storage.dart:46:11)

#6 _registerServices (file:///Users/zb/pub-dartlang-dart/app/bin/service/analyzer.dart:92:27)

#7 _frontendMain. (file:///Users/zb/pub-dartlang-dart/app/bin/service/analyzer.dart:58:11)

#8 _withInmemoryCache (package:pub_dartlang_org/shared/redis_cache.dart:70:18)

#9 withAppEngineAndCache. (package:pub_dartlang_org/shared/redis_cache.dart:38:20)

#10 withAppEngineServices. (package:appengine/src/appengine_internal.dart:42:51)
#11 _withAppEngineServicesInternal. (package:appengine/src/appengine_internal.dart:110:20)

#12 _ServiceScope._fork. (package:gcloud/service_scope.dart:210:19)
#13 _rootRun (dart:async/zone.dart:1124:13)
#14 _CustomZone.run (dart:async/zone.dart:1021:19)
#15 _runZoned (dart:async/zone.dart:1516:10)
#16 runZoned (dart:async/zone.dart:1463:12)
#17 _ServiceScope._fork (package:gcloud/service_scope.dart:209:12)
#18 fork (package:gcloud/service_scope.dart:104:30)
#19 _withAppEngineServicesInternal (package:appengine/src/appengine_internal.dart:101:10)
#20 withAppEngineServices (package:appengine/src/appengine_internal.dart:42:5)
#21 withAppEngineServices (package:appengine/appengine.dart:136:10)
#22 withAppEngineAndCache (package:pub_dartlang_org/shared/redis_cache.dart:32:10)

#23 _frontendMain (file:///Users/zb/pub-dartlang-dart/app/bin/service/analyzer.dart:57:9)

#24 _wrapper. (package:pub_dartlang_org/shared/service_utils.dart:226:22)

#25 Chain.capture. (package:stack_trace/src/chain.dart:101:24)
#26 _rootRun (dart:async/zone.dart:1124:13)
#27 _CustomZone.run (dart:async/zone.dart:1021:19)
#28 _runZoned (dart:async/zone.dart:1516:10)
#29 runZoned (dart:async/zone.dart:1463:12)
#30 Chain.capture (package:stack_trace/src/chain.dart:99:12)
#31 _wrapper (package:pub_dartlang_org/shared/service_utils.dart:224:9)
#32 _startIsolate. (dart:isolate/runtime/libisolate_patch.dart:303:17)
#33 _RawReceivePortImpl._handleMessage (dart:isolate/runtime/libisolate_patch.dart:171:12)

2019-03-21 15:35:54.977167 Error: pub.popularity: Unable to load popularity data: gs://dart-app-235113--popularity/v3/popularity.json.gz

Error:
DetailedApiRequestError(status: 404, message: No error details. HTTP status was: 404.)

Stack:
#0 _validateResponse (package:_discoveryapis_commons/src/clients.dart:879:5)

#1 ApiRequester.request (package:_discoveryapis_commons/src/clients.dart:69:22)

#2 ObjectsResourceApi.get (package:googleapis/storage/v1.dart:2832:32)
#3 _BucketImpl.read (file:///Users/zb/.pub-cache/hosted/pub.dartlang.org/gcloud-0.6.0+2/lib/src/storage_impl.dart:224:10)

#4 PopularityStorage.fetch (package:pub_dartlang_org/shared/popularity_storage.dart:62:12)

#5 PopularityStorage.init (package:pub_dartlang_org/shared/popularity_storage.dart:46:11)

#6 _registerServices (file:///Users/zb/pub-dartlang-dart/app/bin/service/analyzer.dart:92:27)

#7 _workerMain. (file:///Users/zb/pub-dartlang-dart/app/bin/service/analyzer.dart:69:11)

#8 _withInmemoryCache (package:pub_dartlang_org/shared/redis_cache.dart:70:18)

#9 withAppEngineAndCache. (package:pub_dartlang_org/shared/redis_cache.dart:38:20)

#10 withAppEngineServices. (package:appengine/src/appengine_internal.dart:42:51)
#11 _withAppEngineServicesInternal. (package:appengine/src/appengine_internal.dart:110:20)

#12 _ServiceScope._fork. (package:gcloud/service_scope.dart:210:19)
#13 _rootRun (dart:async/zone.dart:1124:13)
#14 _CustomZone.run (dart:async/zone.dart:1021:19)
#15 _runZoned (dart:async/zone.dart:1516:10)
#16 runZoned (dart:async/zone.dart:1463:12)
#17 _ServiceScope._fork (package:gcloud/service_scope.dart:209:12)
#18 fork (package:gcloud/service_scope.dart:104:30)
#19 _withAppEngineServicesInternal (package:appengine/src/appengine_internal.dart:101:10)
#20 withAppEngineServices (package:appengine/src/appengine_internal.dart:42:5)
#21 withAppEngineServices (package:appengine/appengine.dart:136:10)
#22 withAppEngineAndCache (package:pub_dartlang_org/shared/redis_cache.dart:32:10)

#23 _workerMain (file:///Users/zb/pub-dartlang-dart/app/bin/service/analyzer.dart:68:9)

#24 _wrapper. (package:pub_dartlang_org/shared/service_utils.dart:226:22)

#25 Chain.capture. (package:stack_trace/src/chain.dart:101:24)
#26 _rootRun (dart:async/zone.dart:1124:13)
#27 _CustomZone.run (dart:async/zone.dart:1021:19)
#28 _runZoned (dart:async/zone.dart:1516:10)
#29 runZoned (dart:async/zone.dart:1463:12)
#30 Chain.capture (package:stack_trace/src/chain.dart:99:12)
#31 _wrapper (package:pub_dartlang_org/shared/service_utils.dart:224:9)
#32 _startIsolate. (dart:isolate/runtime/libisolate_patch.dart:303:17)
#33 _RawReceivePortImpl._handleMessage (dart:isolate/runtime/libisolate_patch.dart:171:12)

2019-03-21 15:35:56.335620 Error: pub.job: analyzer job error [pull]

Error:
GrpcError: "Missing or insufficient permissions." (code: 7)

Stack:
#0 Client.invoke (package:appengine/src/grpc_api_impl/grpc.dart:221:9)

#1 Channel.invoke (package:appengine/src/grpc_api_impl/grpc.dart:45:19)

#2 DatastoreApi.runQuery (package:appengine/src/grpc_api/dart/google/datastore/v1/datastore.pb.dart:1021:20)
#3 GrpcDatastoreImpl.query (package:appengine/src/grpc_api_impl/datastore_impl.dart:252:32)

#4 Query.run. (file:///Users/zb/.pub-cache/hosted/pub.dartlang.org/gcloud-0.6.0+2/lib/src/db/db.dart:239:12)
#5 StreamFromPages._onListen (package:gcloud/common.dart:72:5)
#6 _runGuarded (dart:async/stream_controller.dart:805:24)
#7 _StreamController._subscribe. (dart:async/stream_controller.dart:684:7)
#8 _BufferingStreamSubscription._guardCallback (dart:async/stream_impl.dart:414:13)
#9 _StreamController._subscribe (dart:async/stream_controller.dart:683:18)
#10 _ControllerStream._createSubscription (dart:async/stream_controller.dart:818:19)
#11 _StreamImpl.listen (dart:async/stream_impl.dart:472:9)
#12 new _ForwardingStreamSubscription (dart:async/stream_pipe.dart:123:10)
#13 _ForwardingStream._createSubscription (dart:async/stream_pipe.dart:91:16)
#14 _ForwardingStream.listen (dart:async/stream_pipe.dart:86:12)
#15 Stream.toList (dart:async/stream.dart:969:10)
#16 JobBackend.lockAvailable (package:pub_dartlang_org/job/backend.dart:150:36)

#17 JobProcessor.run (package:pub_dartlang_org/job/job.dart:47:30)

#18 JobMaintenance.run (package:pub_dartlang_org/job/job.dart:95:18)
#19 _workerMain. (file:///Users/zb/pub-dartlang-dart/app/bin/service/analyzer.dart:83:26)
#20 _asyncThenWrapperHelper. (dart:async/runtime/libasync_patch.dart:77:64)
#21 StackZoneSpecification._registerUnaryCallback.. (package:stack_trace/src/stack_zone_specification.dart:129:26)
#22 StackZoneSpecification._run (package:stack_trace/src/stack_zone_specification.dart:209:15)
#23 StackZoneSpecification._registerUnaryCallback. (package:stack_trace/src/stack_zone_specification.dart:129:14)
#24 _rootRunUnary (dart:async/zone.dart:1132:38)
#25 _CustomZone.runUnary (dart:async/zone.dart:1029:19)
#26 _FutureListener.handleValue (dart:async/future_impl.dart:126:18)
#27 Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:639:45)
#28 Future._propagateToListeners (dart:async/future_impl.dart:668:32)
#29 Future._complete (dart:async/future_impl.dart:473:7)
#30 _SyncCompleter.complete (dart:async/future_impl.dart:51:12)
#31 _AsyncAwaitCompleter.complete (dart:async/runtime/libasync_patch.dart:28:18)
#32 _completeOnAsyncReturn (dart:async/runtime/libasync_patch.dart:294:13)
#33 _registerServices (file:///Users/zb/pub-dartlang-dart/app/bin/service/analyzer.dart)
#34 _asyncThenWrapperHelper. (dart:async/runtime/libasync_patch.dart:77:64)
#35 StackZoneSpecification._registerUnaryCallback.. (package:stack_trace/src/stack_zone_specification.dart:129:26)
#36 StackZoneSpecification._run (package:stack_trace/src/stack_zone_specification.dart:209:15)
#37 StackZoneSpecification._registerUnaryCallback. (package:stack_trace/src/stack_zone_specification.dart:129:14)
#38 _rootRunUnary (dart:async/zone.dart:1132:38)
#39 _CustomZone.runUnary (dart:async/zone.dart:1029:19)
#40 _FutureListener.handleValue (dart:async/future_impl.dart:126:18)
#41 Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:639:45)
#42 Future._propagateToListeners (dart:async/future_impl.dart:668:32)
#43 Future._complete (dart:async/future_impl.dart:473:7)
#44 _SyncCompleter.complete (dart:async/future_impl.dart:51:12)
#45 _AsyncAwaitCompleter.complete (dart:async/runtime/libasync_patch.dart:28:18)
#46 _completeOnAsyncReturn (dart:async/runtime/libasync_patch.dart:294:13)
#47 getOrCreateBucket (package:pub_dartlang_org/shared/storage.dart)
#48 _asyncThenWrapperHelper. (dart:async/runtime/libasync_patch.dart:77:64)
#49 StackZoneSpecification._registerUnaryCallback.. (package:stack_trace/src/stack_zone_specification.dart:129:26)
#50 StackZoneSpecification._run (package:stack_trace/src/stack_zone_specification.dart:209:15)
#51 StackZoneSpecification._registerUnaryCallback. (package:stack_trace/src/stack_zone_specification.dart:129:14)
#52 _rootRunUnary (dart:async/zone.dart:1132:38)
#53 _CustomZone.runUnary (dart:async/zone.dart:1029:19)

@jonasfj
Copy link
Member

jonasfj commented Mar 21, 2019

2019-03-21 15:35:54.977167 Error: pub.popularity: Unable to load popularity data: gs://dart-app-235113--popularity/v3/popularity.json.gz

Is an attempt to load popularity data... I think it'll ignore it if not present...

GrpcError: "Missing or insufficient permissions." (code: 7)

suggests that you haven't got permissions for datastore, maybe you need to enable the API.


Note. we don't support running custom deployments of pub-dartlang-dart, instructions for how to run it mostly for local development, so you're a bit on your own here.

You're ofcourse welcome to help out with development by filing PRs and fixing bugs... but if you're mostly interested in running a custom pub server, I would suggest looking at @v7lin's simple_pub_server :)

@jonasfj jonasfj closed this as completed Mar 21, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants