Skip to content

Commit

Permalink
feat(aws_savings_plan#Understanding how reserved instances are applie…
Browse files Browse the repository at this point in the history
…d): Understanding how reserved instances are applied

A Reserved Instance that is purchased for a Region is called a regional Reserved Instance, and provides Availability Zone and instance size flexibility.

- The Reserved Instance discount applies to instance usage in any Availability Zone in that Region.
- The Reserved Instance discount applies to instance usage within the instance family, regardless of size—this is known as instance size flexibility.

With instance size flexibility, the Reserved Instance discount applies to instance usage for instances that have the same family, generation, and attribute. The Reserved Instance is applied from the smallest to the largest instance size within the instance family based on the normalization factor.

The discount applies either fully or partially to running instances of the same instance family, depending on the instance size of the reservation, in any Availability Zone in the Region. The only attributes that must be matched are the instance family, tenancy, and platform.

The following table lists the different sizes within an instance family, and the corresponding normalization factor. This scale is used to apply the discounted rate of Reserved Instances to the normalized usage of the instance family.

| Instance size | 	Normalization factor |
| --- | --- |
| nano | 	0.25 |
| micro | 	0.5 |
| small | 	1 |
| medium | 	2 |
| large | 	4 |
| xlarge | 	8 |
| 2xlarge | 	16 |
| 3xlarge | 	24 |
| 4xlarge | 	32 |
| 6xlarge | 	48 |
| 8xlarge | 	64 |
| 9xlarge | 	72 |
| 10xlarge | 	80 |
| 12xlarge | 	96 |
| 16xlarge | 	128 |
| 18xlarge | 	144 |
| 24xlarge | 	192 |
| 32xlarge | 	256 |
| 48xlarge | 	384 |
| 56xlarge | 	448 |
| 112xlarge | 	896 |

For example, a `t2.medium` instance has a normalization factor of `2`. If you purchase a `t2.medium` default tenancy Amazon Linux/Unix Reserved Instance in the US East (N. Virginia) and you have two running `t2.small` instances in your account in that Region, the billing benefit is applied in full to both instances.

Or, if you have one `t2.large` instance running in your account in the US East (N. Virginia) Region, the billing benefit is applied to 50% of the usage of the instance.

Limitations:

- *Supported*: Instance size flexibility is only supported for Regional Reserved Instances.
- *Not supported*: Instance size flexibility is not supported for the following Reserved Instances:
    - Reserved Instances that are purchased for a specific Availability Zone (zonal Reserved Instances)
    - Reserved Instances for G4ad, G4dn, G5, G5g, and Inf1 instances
    - Reserved Instances for Windows Server, Windows Server with SQL Standard, Windows Server with SQL Server Enterprise, Windows Server with SQL Server Web, RHEL, and SUSE Linux Enterprise Server
    - Reserved Instances with dedicated tenancy

feat(gpu): Introduce GPU

[GPU](https://en.wikipedia.org/wiki/Graphics_processing_unit) or Graphic Processing Unit is a specialized electronic circuit initially designed to accelerate computer graphics and image processing (either on a video card or embedded on motherboards, mobile phones, personal computers, workstations, and game consoles).

For years I've wanted to buy a graphic card but I've been stuck in the problem that I don't have a desktop. I have a X280 lenovo laptop used to work and personal use with an integrated card that has let me so far to play old games such as [King Arthur Gold](kag.md) or [Age of Empires II](age_of_empires.md), but has hard times playing "newer" games such as It takes two. Last year I also bought a [NAS](nas.md) with awesome hardware. So it makes no sense to buy a desktop just for playing.

Now that I host [Jellyfin](jellyfin.md) on the NAS and that machine learning is on the hype with a lot of interesting solutions that can be self-hosted (whisper, chatgpt similar solutions...), it starts to make sense to add a GPU to the server. What made me give the step is that you can also self-host a gaming server to stream to any device! It makes so much sense to have all the big guns inside the NAS and stream the content to the less powerful devices.

That way if you host services, you make the most use of the hardware.

feat(grapheneos#Disable Bluetooth and Wifi once it's disconnected): Disable Bluetooth and Wifi once it's disconnected

If you don't want to go spreading your SSIDs you can configure graphene to disable wifi and bluetooth X minutes after loosing connection.

For Wifi:

- Go to Settings > network & internet > internet > network preferences
- Select Turn off Wi-Fi automatically, for example after 2 minutes

For Bluetooth:

- Go to Settings > connected devices > connection preferences>  bluetooth
- Select Turn Bluetooth off automatically, for example after 2 minutes

feat(moonlight): Introduce moonlight

[Moonlight](https://github.com/moonlight-stream/moonlight-docs/wiki) is an open source client implementation of NVIDIA GameStream that allows you to to stream your collection of games and apps from your GameStream-compatible PC to another device on your network or the Internet. You can play your favorite games on your PC, phone, tablet, or TV with Moonlight..

References:

- [Home](https://moonlight-stream.org/)
- [Docs](https://github.com/moonlight-stream/moonlight-docs/wiki)

feat(sqlite#Order by a column descending): Order by a column descending

```sql
SELECT
   select_list
FROM
   table
ORDER BY
    column_1 ASC,
    column_2 DESC;
```

feat(promtail): Introduce Promtail

[Promtail](https://grafana.com/docs/loki/latest/send-data/promtail/) is an agent which ships the contents of local logs to a [Loki](loki.md) instance.

It is usually deployed to every machine that runs applications which need to be monitored.

It primarily:

- Discovers targets
- Attaches labels to log streams
- Pushes them to the Loki instance.
  • Loading branch information
lyz-code committed Jan 12, 2024
1 parent 48281fe commit 389ce8c
Show file tree
Hide file tree
Showing 12 changed files with 7,641 additions and 7 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
site
.pdm.toml
.pdm-python
125 changes: 125 additions & 0 deletions docs/aws_savings_plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,129 @@ Savings Plan rate is $2.40. This is less than the $3.00/hour commitment.
Next, the Compute Savings Plan is applied to rest of the resource usage, if it
doesn't cover the whole expense, then On demand rates will apply.

# [Understanding how reserved instances are applied](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/apply_ri.html)

A Reserved Instance that is purchased for a Region is called a regional Reserved Instance, and provides Availability Zone and instance size flexibility.

- The Reserved Instance discount applies to instance usage in any Availability Zone in that Region.
- The Reserved Instance discount applies to instance usage within the instance family, regardless of size—this is known as instance size flexibility.

With instance size flexibility, the Reserved Instance discount applies to instance usage for instances that have the same family, generation, and attribute. The Reserved Instance is applied from the smallest to the largest instance size within the instance family based on the normalization factor.

The discount applies either fully or partially to running instances of the same instance family, depending on the instance size of the reservation, in any Availability Zone in the Region. The only attributes that must be matched are the instance family, tenancy, and platform.

The following table lists the different sizes within an instance family, and the corresponding normalization factor. This scale is used to apply the discounted rate of Reserved Instances to the normalized usage of the instance family.

| Instance size | Normalization factor |
| --- | --- |
| nano | 0.25 |
| micro | 0.5 |
| small | 1 |
| medium | 2 |
| large | 4 |
| xlarge | 8 |
| 2xlarge | 16 |
| 3xlarge | 24 |
| 4xlarge | 32 |
| 6xlarge | 48 |
| 8xlarge | 64 |
| 9xlarge | 72 |
| 10xlarge | 80 |
| 12xlarge | 96 |
| 16xlarge | 128 |
| 18xlarge | 144 |
| 24xlarge | 192 |
| 32xlarge | 256 |
| 48xlarge | 384 |
| 56xlarge | 448 |
| 112xlarge | 896 |

For example, a `t2.medium` instance has a normalization factor of `2`. If you purchase a `t2.medium` default tenancy Amazon Linux/Unix Reserved Instance in the US East (N. Virginia) and you have two running `t2.small` instances in your account in that Region, the billing benefit is applied in full to both instances.

Or, if you have one `t2.large` instance running in your account in the US East (N. Virginia) Region, the billing benefit is applied to 50% of the usage of the instance.

Limitations:

- *Supported*: Instance size flexibility is only supported for Regional Reserved Instances.
- *Not supported*: Instance size flexibility is not supported for the following Reserved Instances:
- Reserved Instances that are purchased for a specific Availability Zone (zonal Reserved Instances)
- Reserved Instances for G4ad, G4dn, G5, G5g, and Inf1 instances
- Reserved Instances for Windows Server, Windows Server with SQL Standard, Windows Server with SQL Server Enterprise, Windows Server with SQL Server Web, RHEL, and SUSE Linux Enterprise Server
- Reserved Instances with dedicated tenancy

## [Examples of applying reserved instances](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/apply_ri.html)

### Scenario 1: Reserved Instances in a single account

You are running the following On-Demand Instances in account A:

- 4 x `m3.large` Linux, default tenancy instances in Availability Zone us-east-1a
- 2 x `m4.xlarge` Amazon Linux, default tenancy instances in Availability Zone us-east-1b
- 1 x `c4.xlarge` Amazon Linux, default tenancy instances in Availability Zone us-east-1c

You purchase the following Reserved Instances in account A:

- 4 x `m3.large` Linux, default tenancy Reserved Instances in Availability Zone us-east-1a (capacity is reserved)
- 4 x `m4.large` Amazon Linux, default tenancy Reserved Instances in Region us-east-1
- 1 x `c4.large` Amazon Linux, default tenancy Reserved Instances in Region us-east-1

The Reserved Instance benefits are applied in the following way:

- The discount and capacity reservation of the four `m3.large` zonal Reserved Instances is used by the four `m3.large` instances because the attributes (instance size, Region, platform, tenancy) between them match.
- The `m4.large` regional Reserved Instances provide Availability Zone and instance size flexibility, because they are regional Amazon Linux Reserved Instances with default tenancy.

An `m4.large` is equivalent to 4 normalized units/hour.

You've purchased four `m4.large` regional Reserved Instances, and in total, they are equal to `16` normalized units/hour (4x4). Account A has two `m4.xlarge` instances running, which is equivalent to `16` normalized units/hour (2x8). In this case, the four `m4.large` regional Reserved Instances provide the full billing benefit to the usage of the two `m4.xlarge` instances.

The `c4.large` regional Reserved Instance in us-east-1 provides Availability Zone and instance size flexibility, because it is a regional Amazon Linux Reserved Instance with default tenancy, and applies to the `c4.xlarge` instance. A `c4.large` instance is equivalent to `4` normalized units/hour and a `c4.xlarge` is equivalent to `8` normalized units/hour.

In this case, the `c4.large` regional Reserved Instance provides partial benefit to `c4.xlarge` usage. This is because the `c4.large` Reserved Instance is equivalent to `4` normalized units/hour of usage, but the `c4.xlarge` instance requires `8` normalized units/hour. Therefore, the `c4.large` Reserved Instance billing discount applies to 50% of `c4.xlarge` usage. The remaining `c4.xlarge` usage is charged at the On-Demand rate.

### Scenario 2: Reserved Instances in a single account using the normalization factor

You are running the following On-Demand Instances in account A:

- 2 x `m3.xlarge` Amazon Linux, default tenancy instances in Availability Zone us-east-1a
- 2 x `m3.large` Amazon Linux, default tenancy instances in Availability Zone us-east-1b

You purchase the following Reserved Instance in account A:

- 1 x `m3.2xlarge` Amazon Linux, default tenancy Reserved Instance in Region us-east-1

The Reserved Instance benefits are applied in the following way:

- The `m3.2xlarge` regional Reserved Instance in us-east-1 provides Availability Zone and instance size flexibility, because it is a regional Amazon Linux Reserved Instance with default tenancy. It applies first to the `m3.large` instances and then to the `m3.xlarge` instances, because it applies from the smallest to the largest instance size within the instance family based on the normalization factor.

- An `m3.large` instance is equivalent to `4` normalized units/hour.
- An `m3.xlarge` instance is equivalent to `8` normalized units/hour.
- An `m3.2xlarge` instance is equivalent to `16` normalized units/hour.

The benefit is applied as follows:

The `m3.2xlarge` regional Reserved Instance provides full benefit to 2 x `m3.large` usage, because together these instances account for `8` normalized units/hour. This leaves `8` normalized units/hour to apply to the `m3.xlarge` instances.

With the remaining `8` normalized units/hour, the `m3.2xlarge` regional Reserved Instance provides full benefit to 1 x `m3.xlarge` usage, because each `m3.xlarge` instance is equivalent to `8` normalized units/hour. The remaining `m3.xlarge` usage is charged at the On-Demand rate.

# [Standard or convertible reserved instances](https://docs.aws.amazon.com/whitepapers/latest/cost-optimization-reservation-models/standard-vs.-convertible-offering-classes.html)

When you purchase a Reserved Instance, you can choose between a Standard or Convertible offering class.

- Standard Reserved Instance: Enables you to modify Availability Zone, scope, networking type, and instance size (within the same instance type) of your Reserved Instance.
- Convertible Reserved Instance: Enables you to exchange one or more Convertible Reserved Instances for another Convertible Reserved Instance with a different configuration, including instance family, operating system, and tenancy.

There are no limits to how many times you perform an exchange, as long as the target Convertible Reserved Instance is of an equal or higher value than the Convertible Reserved Instances that you are exchanging.

Standard Reserved Instances typically provide the highest discount levels. One-year Standard Reserved Instances provide a similar discount to three-year Convertible Reserved Instances.

Convertible Reserved Instances are useful when:

- Purchasing Reserved Instances in the payer account instead of a subaccount. You can more easily modify Convertible Reserved Instances to meet changing needs across your organization.
- Workloads are likely to change. In this case, a Convertible Reserved Instance enables you to adapt as needs evolve while still obtaining discounts and capacity reservations.
- You want to hedge against possible future price drops.
- You can’t or don’t want to ask teams to do capacity planning or forecasting.
- You expect compute usage to remain at the committed amount over the commitment period.

# [Monitoring the savings plan](https://docs.aws.amazon.com/savingsplans/latest/userguide/sp-monitoring.html)

Monitoring is an important part of your Savings Plans usage. Understanding the
Expand Down Expand Up @@ -222,6 +345,8 @@ Plans. You can monitor your usage in multiple forms.

* "Reservation covered hours": the column does not refer to your RI hours. This column refers to your on demand hours that was covered by Reserved Instances.

- Using the Reserved Instance Coverage report: In this report the column "Reservation covered hours" is also misleading as it does not refer to your Reserved instance hours. It refers to your on demand hours that was covered by Reserved Instances.

# Doing your savings plan

Go to the [AWS savings plan
Expand Down
2 changes: 1 addition & 1 deletion docs/devops/aws/aws.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,4 @@ aws ec2 stop-instances --instance-ids i-xxxxxxxx

# References

- [Compare ec2 instance types](https://instances.vantage.sh/)
- [Compare aws ec2 instance types and prices](https://instances.vantage.sh/)
32 changes: 32 additions & 0 deletions docs/gpu.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
[GPU](https://en.wikipedia.org/wiki/Graphics_processing_unit) or Graphic Processing Unit is a specialized electronic circuit initially designed to accelerate computer graphics and image processing (either on a video card or embedded on motherboards, mobile phones, personal computers, workstations, and game consoles).

For years I've wanted to buy a graphic card but I've been stuck in the problem that I don't have a desktop. I have a X280 lenovo laptop used to work and personal use with an integrated card that has let me so far to play old games such as [King Arthur Gold](kag.md) or [Age of Empires II](age_of_empires.md), but has hard times playing "newer" games such as It takes two. Last year I also bought a [NAS](nas.md) with awesome hardware. So it makes no sense to buy a desktop just for playing.

Now that I host [Jellyfin](jellyfin.md) on the NAS and that machine learning is on the hype with a lot of interesting solutions that can be self-hosted (whisper, chatgpt similar solutions...), it starts to make sense to add a GPU to the server. What made me give the step is that you can also self-host a gaming server to stream to any device! It makes so much sense to have all the big guns inside the NAS and stream the content to the less powerful devices.

That way if you host services, you make the most use of the hardware.

# Market analysis

Done on January of 2024 taking into account the next requirements:

- Price under 600$
- Able to perform the next actions for at least 5 years:
- [Jellyfin](jellyfin.md) transcoding
- Videogame streaming from a headless server.
- Machine learning operations.
-

at I don't wa

Using these sources:

- [PCGamer 2024 review](https://www.pcgamer.com/the-best-graphics-cards/)
- [Gamesradar 2024 review](https://www.gamesradar.com/best-pc-graphics-cards/)

## Overview of the market analysis

The best graphics card is objectively Nvidia's RTX 4090 but it's too expensive. Then there's the RTX 4080, which is a bit too pricey for us, and the RTX 4070 Ti. The RTX 4070 Ti also costs a heap more cash than we'd like, but at least it's more reasonable than Nvidia's finest for a perfectly 4K capable card.

On the other end of the market, Nvidia has a rather uninspired upgrade in the RTX 4060. We also met the release of AMD's RX 7600 with a shrug, but at least it's cheap enough now to feel more competitive. And Intel still has a dog in the budget game: the Arc A750. When this card drops down to around $200, it's a steal, though the drivers aren't always up to the standard we'd like to see. That leaves AMD's RX 7600 as the best budget graphics card today, mostly for being a boringly safe pick.

14 changes: 14 additions & 0 deletions docs/grapheneos.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,20 @@ Attestation can be done locally by pairing with another Android 8+ device or rem

# Tips

## Disable Bluetooth and Wifi once it's disconnected

If you don't want to go spreading your SSIDs you can configure graphene to disable wifi and bluetooth X minutes after loosing connection.

For Wifi:

- Go to Settings > network & internet > internet > network preferences
- Select Turn off Wi-Fi automatically, for example after 2 minutes

For Bluetooth:

- Go to Settings > connected devices > connection preferences> bluetooth
- Select Turn Bluetooth off automatically, for example after 2 minutes

## [Split the screen](https://www.reddit.com/r/GrapheneOS/comments/134iqr3/split_screen/)

Go into app switcher, tap on the app icon above the active app and then select "Split top".
Expand Down
163 changes: 159 additions & 4 deletions docs/loki.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,169 @@ A small index and highly compressed chunks simplifies the operation and signific

# [Installation](https://grafana.com/docs/loki/latest/setup/install/docker/)

There are [many ways to install Loki](https://grafana.com/docs/loki/latest/setup/install/), we're going to do it using `docker-compose` taking [their example as a starting point](https://raw.githubusercontent.com/grafana/loki/v2.9.1/production/docker-compose.yaml) and complementing our already existent [grafana docker-compose](grafana.md#installation).
There are [many ways to install Loki](https://grafana.com/docs/loki/latest/setup/install/), we're going to do it using `docker-compose` taking [their example as a starting point](https://raw.githubusercontent.com/grafana/loki/v2.9.1/production/docker-compose.yaml) and complementing our already existent [grafana docker-compose](grafana.md#installation). It uses [the official loki docker](https://hub.docker.com/r/grafana/loki)

## Set up the docker compose

Save the next docker compose at `/data/grafana` or wherever you want:

```yaml
---
version: "3.3"
services:
grafana:
image: grafana/grafana-oss:${GRAFANA_VERSION:-latest}
container_name: grafana
restart: unless-stopped
volumes:
- grafana-config:/etc/grafana
- grafana-data:/var/lib/grafana
networks:
- grafana
- swag
env_file:
- .env
depends_on:
- db
db:
image: postgres:${DATABASE_VERSION:-15}
restart: unless-stopped
container_name: grafana-db
environment:
- POSTGRES_DB=${GF_DATABASE_NAME:-grafana}
- POSTGRES_USER=${GF_DATABASE_USER:-grafana}
- POSTGRES_PASSWORD=${GF_DATABASE_PASSWORD:?database password required}
networks:
- grafana
volumes:
- db-data:/var/lib/postgresql/data
env_file:
- .env

loki:
image: grafana/loki:${LOKI_VERSION:-latest}
container_name: loki
ports:
- "3100:3100"
command: -config.file=/etc/loki/local-config.yaml -print-config-stderr
volumes:
- loki-data:/loki
networks:
- grafana
- loki
env_file:
- .env

promtail:
image: grafana/promtail:${PROMTAIL_VERSION:-latest}
container_name: promtail
volumes:
- /var/log:/var/log
command: -config.file=/etc/promtail/config.yml
networks:
- loki
env_file:
- .env
networks:
grafana:
external:
name: grafana
swag:
external:
name: swag
loki:
external:
name: loki

volumes:
grafana-config:
driver: local
driver_opts:
type: none
o: bind
device: /data/grafana/app/config
grafana-data:
driver: local
driver_opts:
type: none
o: bind
device: /data/grafana/app/data
db-data:
driver: local
driver_opts:
type: none
o: bind
device: /data/grafana/database
loki-data:
driver: local
driver_opts:
type: none
o: bind
device: /data/loki/data
```
Define the `.env` file similar to:

```env
# Check all configuration options at:
# https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana
# -----------------------------
# --- General configuration ---
# -----------------------------
GRAFANA_VERSION=latest
GF_DEFAULT_INSTANCE_NAME="production"
GF_SERVER_ROOT_URL="https://production.your-domain.org"
# Set this option to true to enable HTTP compression, this can improve transfer
# speed and bandwidth utilization. It is recommended that most users set it to
# true. By default it is set to false for compatibility reasons.
GF_SERVER_ENABLE_GZIP="true"
# ------------------------------
# --- Database configuration ---
# ------------------------------
DATABASE_VERSION=15
GF_DATABASE_TYPE=postgres
GF_DATABASE_HOST=grafana-db:5432
GF_DATABASE_NAME=grafana
GF_DATABASE_USER=grafana
GF_DATABASE_PASSWORD="your-super-secret-password"
GF_DATABASE_SSL_MODE=disable
GF_DATABASE_LOG_QUERIES="false"
# --------------------------
# --- Loki configuration ---
# --------------------------
LOKI_VERSION=latest
# ------------------------------
# --- Promtail configuration ---
# ------------------------------
PROMTAIL_VERSION=latest
```

## Configure Loki

Download and edit [the basic configuration](https://grafana.com/docs/loki/latest/setup/install/docker/)

```bash
wget https://raw.githubusercontent.com/grafana/loki/v2.9.1/cmd/loki/loki-local-config.yaml -O /data/loki/config/loki-config.yaml
```

## Configure Promtail

Download and edit [the basic configuration](https://grafana.com/docs/loki/latest/setup/install/docker/)

```bash
wget https://raw.githubusercontent.com/grafana/loki/v2.9.1/clients/cmd/promtail/promtail-docker-config.yaml -O /data/promtail/promtail-config.yaml
```

## Configure Grafana

It makes use of the [environment variables to configure Loki](https://grafana.com/docs/loki/latest/configure/#configuration-file-reference), that's why we have the `-config.expand-env=true` flag in the command line launch.

In the grafana datasources directory add `loki.yaml`:
Expand Down Expand Up @@ -39,9 +197,6 @@ Loki 2.0 brings an index mechanism named ‘boltdb-shipper’ and is what we now

Loki 2.8 adds TSDB as a new mode for the Single Store and is now the recommended way to persist data in Loki as it improves query performance, reduces TCO and has the same feature parity as “boltdb-shipper”.

# Usage

## [Build dashboards](https://grafana.com/blog/2020/04/08/loki-quick-tip-how-to-create-a-grafana-dashboard-for-searching-logs-using-loki-and-prometheus/)
Expand Down
6 changes: 6 additions & 0 deletions docs/moonlight.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[Moonlight](https://github.com/moonlight-stream/moonlight-docs/wiki) is an open source client implementation of NVIDIA GameStream that allows you to to stream your collection of games and apps from your GameStream-compatible PC to another device on your network or the Internet. You can play your favorite games on your PC, phone, tablet, or TV with Moonlight..

# References

- [Home](https://moonlight-stream.org/)
- [Docs](https://github.com/moonlight-stream/moonlight-docs/wiki)
Loading

0 comments on commit 389ce8c

Please sign in to comment.