Merge branch 'master' into flypkgs-builds

Author: michaeldwan

.gitignore

@@ -37,3 +37,4 @@ out
 
 .direnv/
 /ci-preflight-test-results.njson
+.tool-versions

CONTRIBUTING.md

@@ -0,0 +1,82 @@
+# Development flow
+
+
+## Building
+
+To build `flyctl`, all you need to do is to run `make build` from the root directory. This will build a binary in the `bin/` directory. Alternatively, you can run `go build .`
+
+To run `flyctl`, you can just run the binary you built using `make build`: `./bin/flyctl`. So for example, to update a machine, you can run `go run . m update -a <app_name> <machine_id>`. Alternatively, you can build and run in the same command by running `go run .`, followed by whatever sub-command you want to run. Just note that this will have a slower startup.
+
+
+## Testing
+
+We have two different kinds of tests in `flyctl`, unit tests and integration tests (preflight). It's recommend to write a test for any features added or bug fixes, in order to prevent regressions in the future.
+
+
+### Integration tests
+
+Unit tests are stored in individual files next to the functionality they're testing. For example
+
+`internal/command/secrets/parser_test.go`
+is a test for the secrets parsing code
+
+`internal/command/secrets/parser_test.go`.
+You can run these tests by running `make test` from the root directory.
+
+
+### Preflight
+
+The integration tests, called preflight, are different. They exist to test flyctl functionality on production infra, in order to make sure that entire commands and workflows don't break. Those are located in the `test/preflight` directory.
+
+For outside contributors, **please be warned that running preflight tests creates real apps and machines and will cost real money**. We already run preflight by default on all pull requests, so we recommend just opening up a draft PR instead. If you work at Fly and want to work on preflight tests, go ahead and continue reading.
+
+Before running any preflight test, you must first set some specific environment variables. It's recommended to set them up using [direnv](https://direnv.net/docs/installation.html). First, copy the `.direnv/preflight-example` file to `.direnv/preflight`. Next, modify `FLY_PREFLIGHT_TEST_FLY_ORG` to an organization you make specifically for testing. Don't use your `personal` org. Modify `FLY_PREFLIGHT_TEST_FLY_REGIONS` to have two regions, ideally ones not the closest ones. For example, `"iad sin"`. Finally, set `FLY_PREFLIGHT_TEST_ACCESS_TOKEN` to whatever `fly auth token` outputs.
+
+To run preflight tests, you can just run `make preflight-test`. If you want to run a specific preflight test, run `make preflight-test T=<test_name>`
+
+If you're trying to decide whether to write a unit test, or an integration test for your change, I recommend just writing a preflight test. They're usually simpler to write, and there's a lot more examples of how to write them.
+
+
+
+## Linting
+
+With the trifecta of the development process nearly complete, let's talk about linting. The linter we run is [golangci-lint](https://golangci-lint.run/). It helps with finding potential bugs in programs, as well as helping you follow standard go conventions. To run it locally, just run `golangci-lint --path-prefix=. run`. If you'd like to run all of our [pre-commit lints](https://pre-commit.com/), then run `pre-commit run --all-files`
+
+# Generating the GraphQL Schema
+
+As of writing this, we host our GraphQL schema on `web`, an internal repo that hosts our GQL based API. Unfortunately, that means that outside contributors can't updated the GraphQL schema used by `flyctl`. While there isn't much of a reason why you may want to do so, we're working on automating update the GQL schema in `flyctl`.
+
+Updating the GraphQL schema from web is a manual process at the moment. To do so, `cd` into `web/`, and run `bundle exec rails graphql:schema:idl && cp ./schema.graphql ../flyctl/gql/schema.graphql`, assuming that `flyctl/` is in the same directory as `web/`
+
+
+# Cutting a release
+
+If you have write access to this repo, you can ship a prerelease with:
+
+`scripts/bump_version.sh prerel`
+
+or a full release with:
+
+`scripts/bump_version.sh`
+
+
+# Committing to flyctl
+
+When committing to `flyctl`, there are a few important things to keep in mind:
+
+-   Keep commits small and focused, it helps greatly when reviewing larger PRs
+-   Make sure to use descriptive messages in your commit messages, it also helps future people understand why a change was made.
+-   PRs are squash merged, so please make sure to use descriptive titles
+
+
+## Examples
+
+[This is a bad example of a commit](https://github.com/superfly/flyctl/pull/1809/commits/6f167c858dbd7ae1324632dda9e29072ddde8ad7), it has a large diff and no explanation as to why this change is being made. [This is a great one](https://github.com/superfly/flyctl/commit/2636f47fe91cbe37018926cb0d7d2227a6887086), since it's a small commit, and it's reasoning as well as the context behind the change. Good commit messages also help contributors in the future to understand *why* we did something a certain way.
+
+
+# Further Go reading
+
+Go is a weird language full of a million different pitfalls. If you haven't already, I strongly recommend reading through these articles:
+
+-   <https://go.dev/doc/effective_go> (just a generally great resource)
+-   <https://go.dev/blog/go1.13-errors> (error wrapping specifically is useful for a lot of the functionality we use)

README.md

@@ -98,42 +98,5 @@ There is a simple Powershell script, `winbuild.ps1`, which will run the code gen
 Run `scripts/build-dfly` to build a Docker image from the current branch. Then, use `scripts/dfly` to run it. This assumes you are already
 authenticated to Fly in your local environment.
 
-## Cutting a release
-
-If you have write access to this repo, you can ship a prerelease or full release with:
-
-`scripts/bump_version.sh prerel`
-
-or
-
-`scripts/bump_version.sh`
-
-## Running preflight tests
-
-A preflight suite of integration tests is located under the test/preflight/ directory. It uses a flyctl binary and runs real user scenarios, including deploying apps and dbs, and validates expected behavior.
-
-**Warning**: Real apps will be deployed that cost real money. The test fixture does its best to destroy resources it creates, but sometimes it may fail to delete a resource.
-
-The easiest way to run the preflight tests is:
-
-Copy `.direnv/preflight-example` to `.direnv/preflight` and edit following these guidelines:
-
-* Grab your auth token from `~/.fly/config.yml`
-* Do not use your "personal" org, create an new org (i.e. `flyctl-tests-YOURNAME`)
-* Set 2 regions, ideally not your closest region because it leads
-  to false positives when --region or primary region handling is buggy.
-	Run `fly platform regions` for valid ids.
-
-Finally run the tests:
-
-	make preflight-test
-
-That builds a flyctl binary (just like running `make`), then runs the preflight tests against that binary.
-
-To run a single test:
-
-```
-make preflight-test T=TestAppsV2Example
-```
-
-Oh, add more preflight tests at `tests/preflight/*`
+## Contributing guide
+See [CONTRIBUTING.md](./CONTRIBUTING.mdl)

gql/generated.go

@@ -68,6 +68,16 @@ enum AccessTokenType {
   """
   pat
 
+  """
+  used for Sentry
+  """
+  sentry
+
+  """
+  access token
+  """
+  token
+
   """
   token generated for our UI frontend
   """
@@ -190,11 +200,6 @@ type AddOn implements Node {
   Status of the add-on
   """
   status: String
-
-  """
-  Token for the add-on
-  """
-  token: String
 }
 
 """
@@ -298,16 +303,11 @@ type AddOnProvider {
   selectName: Boolean!
   selectRegion: Boolean!
   selectReplicaRegions: Boolean!
-  tosAgreement: String!
+  tosAgreement: String
   tosUrl: String
 }
 
 enum AddOnType {
-  """
-  A Logtail log receiver
-  """
-  logtail
-
   """
   A PlanetScale database
   """
@@ -323,6 +323,11 @@ enum AddOnType {
   """
   sentry
 
+  """
+  A Supabase database
+  """
+  supabase
+
   """
   An Upstash Redis database
   """
@@ -435,6 +440,11 @@ input AllocateIPAddressInput {
   """
   region: String
 
+  """
+  The name of the associated service
+  """
+  serviceName: String
+
   """
   The type of IP address to allocate (v4, v6, or private_v6)
   """
@@ -1664,6 +1674,11 @@ input BuildVolumeInput {
   """
   clientMutationId: String
 
+  """
+  compute requirements for volume placement (cpu, mem, gpu, ...)
+  """
+  computeRequirements: JSON
+
   """
   id of host dedication
   """
@@ -5872,6 +5887,7 @@ type LimitedAccessToken implements Node {
   id: ID!
   name: String!
   organization: Organization!
+  profileParams: JSON
   token: String!
   tokenHeader: String
   user: User!
@@ -5957,6 +5973,27 @@ type LogEntry {
   timestamp: ISO8601DateTime!
 }
 
+"""
+Autogenerated input type of LogOut
+"""
+input LogOutInput {
+  """
+  A unique identifier for the client performing the mutation.
+  """
+  clientMutationId: String
+}
+
+"""
+Autogenerated return type of LogOut.
+"""
+type LogOutPayload {
+  """
+  A unique identifier for the client performing the mutation.
+  """
+  clientMutationId: String
+  ok: Boolean!
+}
+
 type LoggedCertificate implements Node {
   cert: String!
   id: ID!
@@ -6905,6 +6942,12 @@ type Mutations {
     """
     input: LockAppInput!
   ): LockAppPayload
+  logOut(
+    """
+    Parameters for LogOut
+    """
+    input: LogOutInput!
+  ): LogOutPayload
   migrateVolume(
     """
     Parameters for MigrateVolume
@@ -7299,7 +7342,7 @@ type Organization implements Node {
   """
   Single sign-on link for the given integration type
   """
-  addOnSsoLink: String!
+  addOnSsoLink: String
 
   """
   List third party integrations associated with an organization

gql/schema.graphql

@@ -8,14 +8,25 @@ import (
 	"github.com/superfly/flyctl/internal/version"
 )
 
-var environment = "development"
+var (
+	buildDate   = "<date>"
+	environment = "development"
+)
 
-func loadBuildTime() error {
-	cachedBuildTime = time.Now()
-	return nil
+func loadBuildTime() (err error) {
+	// Makefile sets proper values for buildDate but bare `go run .` doesn't
+	if buildDate == "<date>" {
+		buildDate = time.Now().Format(time.RFC3339)
+	}
+	cachedBuildTime, err = time.Parse(time.RFC3339, buildDate)
+	return
 }
 
 func loadVersion() error {
-	cachedVersion = version.New(cachedBuildTime, "dev", int(cachedBuildTime.Unix()))
+	// Makefile sets proper values for branchName but bare `go run .` doesn't
+	if branchName == "" {
+		branchName = "dev"
+	}
+	cachedVersion = version.New(cachedBuildTime, branchName, int(cachedBuildTime.Unix()))
 	return nil
 }

internal/buildinfo/env_dev.go

@@ -131,6 +131,11 @@ var CommonFlags = flag.Set{
 		Description: "Maximum number of machines to update concurrently when using the immediate deployment strategy.",
 		Default:     16,
 	},
+	flag.Int{
+		Name:        "volume-initial-size",
+		Description: "The initial size in GB for volumes created on first deploy",
+		Default:     1,
+	},
 	flag.VMSizeFlags,
 }
 
@@ -324,6 +329,7 @@ func deployToMachines(
 		ExcludeRegions:         excludeRegions,
 		OnlyRegions:            onlyRegions,
 		ImmediateMaxConcurrent: flag.GetInt(ctx, "immediate-max-concurrent"),
+		VolumeInitialSize:      flag.GetInt(ctx, "volume-initial-size"),
 	})
 	if err != nil {
 		sentry.CaptureExceptionWithAppInfo(err, "deploy", appCompact)

internal/command/deploy/deploy.go

@@ -107,12 +107,17 @@ func (md *machineDeployment) provisionVolumesOnFirstDeploy(ctx context.Context)
 				continue
 			}
 
-			fmt.Fprintf(md.io.Out, "Creating 1GB volume '%s' for process group '%s'. Use 'fly vol extend' to increase its size\n", m.Source, groupName)
+			fmt.Fprintf(
+				md.io.Out,
+				"Creating a %d GB volume named '%s' for process group '%s'. "+
+					"Use 'fly vol extend' to increase its size\n",
+				md.volumeInitialSize, m.Source, groupName,
+			)
 
 			input := api.CreateVolumeRequest{
 				Name:                m.Source,
 				Region:              groupConfig.PrimaryRegion,
-				SizeGb:              api.Pointer(1),
+				SizeGb:              api.Pointer(md.volumeInitialSize),
 				Encrypted:           api.Pointer(true),
 				HostDedicationId:    md.appConfig.HostDedicationID,
 				ComputeRequirements: md.machineGuest,

internal/command/deploy/deploy_first.go

@@ -54,6 +54,7 @@ type MachineDeploymentArgs struct {
 	ExcludeRegions         map[string]interface{}
 	OnlyRegions            map[string]interface{}
 	ImmediateMaxConcurrent int
+	VolumeInitialSize      int
 }
 
 type machineDeployment struct {
@@ -87,6 +88,7 @@ type machineDeployment struct {
 	excludeRegions         map[string]interface{}
 	onlyRegions            map[string]interface{}
 	immediateMaxConcurrent int
+	volumeInitialSize      int
 }
 
 func NewMachineDeployment(ctx context.Context, args MachineDeploymentArgs) (MachineDeployment, error) {
@@ -147,6 +149,10 @@ func NewMachineDeployment(ctx context.Context, args MachineDeploymentArgs) (Mach
 	if immedateMaxConcurrent < 1 {
 		immedateMaxConcurrent = 1
 	}
+	volumeInitialSize := 1
+	if args.VolumeInitialSize > 0 {
+		volumeInitialSize = args.VolumeInitialSize
+	}
 
 	md := &machineDeployment{
 		apiClient:              apiClient,
@@ -172,6 +178,7 @@ func NewMachineDeployment(ctx context.Context, args MachineDeploymentArgs) (Mach
 		excludeRegions:         args.ExcludeRegions,
 		onlyRegions:            args.OnlyRegions,
 		immediateMaxConcurrent: immedateMaxConcurrent,
+		volumeInitialSize:      volumeInitialSize,
 	}
 	if err := md.setStrategy(); err != nil {
 		return nil, err