You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+82Lines changed: 82 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -64,6 +64,88 @@ The tool can be used as a standard Terraform module, with deployment examples pr
64
64
65
65
See the [functions](https://github.com/provectus/data-quality-gate/tree/main/functions) for further details.
66
66
67
+
## Pricing
68
+
69
+
This solution is completely free because it is open source. However, if you want to integrate it into a live/production environment, there will be associated costs due to its cloud-based nature. These costs can be divided into two parts: the required infrastructure (which you may already have in place, such as VPCs and subnets) and the AWS services necessary for data quality implementation.
70
+
71
+
*Note: All the information provided below has been calculated using the maximum score strategy.*
72
+
#### Pricing for required infrastructure
73
+
74
+
| AWS Service | Approximate monthly cost| Description |
75
+
| ------------- | ------------- | ------------- |
76
+
| AWS S3 and DynamoDB endpoints | - | There is no extra charge for gateway-type endpoints. You only pay for the usage of S3 and DynamoDB itself. |
| AWS ECRs (allure, data_test, reports, notifications) | 7 versions * (865mb + 432mb + 380mb) => 11.3gb * 0.1 rate per gb month= 1.13 USD | allure image size = 865mb, data_test image size = 432mb, reports image size = 380mb, notifications image size = 160mb. For the purpose of our calculations, let's assume we are storing 7 versions of each image. |
79
+
| AWS QuickSight | $7.3 aprx rate per user * 5 = 36.4 USD | Let's assume you have a team consisting of 5 individuals who are interested in the QuickSight data quality dashboard. They frequently check for changes, typically 2-3 times per day. |
80
+
81
+
<u>Monthly total is $59.13 US$ per month</u>
82
+
___
83
+
84
+
#### Pricing for data quality specific infrastructure
85
+
For most of the services used by Data Quality, AWS offers a free-tier supply. Additionally, the costs for these services are typically just a fraction of a cent. To provide further clarity, below you can find a basic cost formula and a few usage examples with cost estimations.
86
+
87
+
We are going to count:
88
+
- number of AWS Lambda runs
89
+
- number of AWS StepFunction transitions
90
+
- web reports AWS EC2 instance running(720 hrs per month)
91
+
92
+
| Description | Formula |
93
+
| ------------- | ------------- |
94
+
| number of AWS Lambda runs for each | (number of data sources * number of changes * work_days_month) * lambda specific rate(depends on lambda duration and memory used) |
95
+
| number of AWS StepFunction transitions | number of lambda runs * 2 |
96
+
97
+
##### Small
98
+
99
+
Let's say we have 1000 data sources and half of them changed every day. Number of runs formula for any lambda is **(1000 data sources * 0.5 changed * 30 days)**
100
+
101
+
| AWS Service | Number of runs | Price |
102
+
| ------------ | -------------- | ------ |
103
+
| AWS Lambda AllureReport | 15000 | $8.33 |
104
+
| AWS Lambda DataTest | 15000 | $67.28 |
105
+
| AWS Lambda Reports | 15000 | $2.08 |
106
+
| AWS StepFunctions | 15000 | $0.65 |
107
+
| AWS EC2 Reports S3 Gateway | 720 hrs | $7.25 |
108
+
109
+
<u>Monthly total: 85.59 US$</u>
110
+
111
+
___
112
+
113
+
##### Medium
114
+
115
+
Let's say we have 10000 data sources and 70% of them changed every day.
116
+
Number of runs formula for any lambda is **(10000 data sources * 0.7 changes * 30 days)**
117
+
118
+
| AWS Service | Number of runs | Price |
119
+
| ------------ | -------------- | ------ |
120
+
| AWS Lambda AllureReport | 210k | $203.33 |
121
+
| AWS Lambda DataTest | 210k | $1028.57 |
122
+
| AWS Lambda Reports | 210k | $115.83 |
123
+
| AWS StepFunctions | 210k | $10.40 |
124
+
| AWS EC2 Reports S3 Gateway | 720 hrs | $7.25 |
125
+
126
+
<u>Monthly total: 1 365.38 US$</u>
127
+
128
+
___
129
+
130
+
##### Large
131
+
132
+
Let's say we have 30000 data sources and all of them changed every day.
133
+
Number of runs formula for any lambda is **(30000 data sources * 1 changes * 30 days)**
134
+
135
+
| AWS Service | Number of runs | Price |
136
+
| ------------ | -------------- | ------ |
137
+
| AWS Lambda AllureReport | 900k | $893.34 |
138
+
| AWS Lambda DataTest | 900k | $4430.06 |
139
+
| AWS Lambda Reports | 900k | $518.33 |
140
+
| AWS StepFunctions | 900k | $44.90 |
141
+
| AWS EC2 Reports S3 Gateway | 720 hrs | $7.25 |
142
+
143
+
<u>Monthly total: 5 893.88 US$</u>
144
+
___
145
+
146
+
**Price per changed data source: 0.006 US$**
147
+
148
+
67
149
## License
68
150
69
151
Apache 2 Licensed. See [LICENSE](https://github.com/provectus/data-quality-gate/tree/main/LICENSE) for full details.
0 commit comments