diff --git a/courses/algorithms/02-analysis/README.md b/courses/algorithms/02-analysis/README.md index e69de29b..14307819 100644 --- a/courses/algorithms/02-analysis/README.md +++ b/courses/algorithms/02-analysis/README.md @@ -0,0 +1,166 @@ +# Algorthm Analysis + +Before starting, lets thin about 3 problems: + +For an array of size N, dont overthink. think and answer: + +1. How many iterations a loop run to find a specific number inside an array? (naively) +2. How many comparisons should I make to find two numbers in an array that sum a specific target? (naively) +3. List all different shuffled arrays we can make? (naively) ex for n==3 123, 132, 213, 231, 312, 321 + +## How to measure an algorithm mathematically? + +???+ example "find a number in a vector" + + ```c++ + int find(vector v, int target) { + // how many iterations? + for (int i = 0; i < v.size(); i++) { + // how many comparisons? + if (v[i] == target) { + return i; + } + } + return -1; + } + ``` + +???+ example "find two numbers sum in an array" + + ```c++ + vector findsum2(vector v, int target) { + // how many outer loop iterations? + for (int i = 0; i < v.size(); i++) { + // how many inner loop iterations? + for (int j = i+1; j < v.size(); j++) { + // how many comparisons? + if (v[i] + v[j] == target) { + return {i, j}; + } + } + } + return {-1, -1}; + } + ``` + Check it out on [leetcode](https://leetcode.com/problems/two-sum/description/). Can you solve it better? + +???+ example "Print all pormutations of an array" + + ```c++ + void generatePermutations(std::vector& vec, int index) { + if (index == vec.size() - 1) { + // Print the current permutation + for (int num : vec) { + std::cout << num << " "; + } + std::cout << std::endl; + return; + } + + // how many swaps for every recursive call? + for (int i = index; i < vec.size(); ++i) { + // Swap the elements at indices index and i + std::swap(vec[index], vec[i]); + + // Recursively generate permutations for the rest of the vector + // How deep this can go? + generatePermutations(vec, index + 1); + + // Backtrack: undo the swap to explore other possibilities + std::swap(vec[index], vec[i]); + } + } + ``` + +Trying to be mathematicaly correct, the number of instructions the first one should be a function similar to this: + +1. $f(n) = a*n + b$ : Where $b$ is the cost of what runs before and after the main loop and $a$ is the cost of the loop. +2. $f(n) = a*n^2 + b*n + c$ : Where $c$ is the cost of what runs before and after the oter loop; $b$ is the cost of the outer loop; and $a$ is the cost of the inner loop; +3. $f(n) = a*n!$ : Where $a$ is the cost of what runs before and after the outer loop; + +To simplify, we remove the constants and the lower order terms: + +1. $$ f(n) = n $$ +2. $$ f(n) = n^2 $$ +3. $$ f(n) = n! $$ + +## Difference between Big O vs Big Theta Θ vs Big Omega Ω Notations + +
+ ![img.png](img.png) +
Source: bigocheatsheet.com
+
+ +### Big O + +- Most used notation; +- Upper bound; +- "never worse than"; +- A real case cannot be faster than it; +- $ 0 <= func <= O $ + +### Big Theta Θ + +- Wrongly stated as average; +- Theta is two-sided; +- Tight bound between 2 constants of the same function +- $ k1*Θ <= func <= k2*Θ $ +- When $N$ goes to infinite, it cannot be faster or slower than it; + +### Honorable mentions + +- **Big Omega Ω**: roughly the oposite of Big O; +- **Little o** and **Little Omega (ω)**. The same concept from the big, but exclude the exact bound; + +
+ ![img_1.png](img_1.png) +
Source: freecodecamp.com
+
+ +## Common Big Os + +!!! note inline end "Log" + + In computer science, when we say log, assume base 2, unless expressely stated; + +| Big O | Name | Example | +| --- | --- | --- | +| O(1) | Constant | sum two numbers | +| O(lg(n)) | Logarithmic | binary search | +| O(n) | Linear | search in an array | +| O(n*lg(n)) | Linearithmic | Merge Sort | +| O(n^c) | Polinomial | match 2 sum | +| O(c^n) | Exponential | brute force password of size n | +| O(n!) | factorial | list all combinations | + +### What is log? + +Log is the inverse of exponentiation. It is the number of times you have to multiply a number by itself to get another number. + +$$ log_b(b^x) = x $$ + +### What is binary search? + +In a binary search, we commonly divide the array in half (base 2), and check if the target is in the left or right half. Then we repeat the process until we find the target or we run out of elements. + +
+ ![linear-vs-binary-search-worst-case.gif](linear-vs-binary-search-worst-case.gif) +
Source: mathwarehouse.com
+
+ +## Common data structures and algorithms + +
+ ![img_2.png](img_2.png) +
Source: bigocheatsheet.com
+
+ +
+ ![img_3.png](img_3.png) +
Source: bigocheatsheet.com
+
+ +## Common Issues and misconceptions + +- Big O and Theta are commonly mixed; +- Hashtables: it is commonly assumed that queries on or being O(1); std:: `` and `` are not the ideal implementation! [Watch this CppCon video](https://www.youtube.com/watch?v=ncHmEUmJZf4) for some deep level insights; diff --git a/courses/algorithms/02-analysis/img.png b/courses/algorithms/02-analysis/img.png new file mode 100644 index 00000000..b56c57b3 Binary files /dev/null and b/courses/algorithms/02-analysis/img.png differ diff --git a/courses/algorithms/02-analysis/img_1.png b/courses/algorithms/02-analysis/img_1.png new file mode 100644 index 00000000..dc5c5f26 Binary files /dev/null and b/courses/algorithms/02-analysis/img_1.png differ diff --git a/courses/algorithms/02-analysis/img_2.png b/courses/algorithms/02-analysis/img_2.png new file mode 100644 index 00000000..e78c5729 Binary files /dev/null and b/courses/algorithms/02-analysis/img_2.png differ diff --git a/courses/algorithms/02-analysis/img_3.png b/courses/algorithms/02-analysis/img_3.png new file mode 100644 index 00000000..c7b2f0e0 Binary files /dev/null and b/courses/algorithms/02-analysis/img_3.png differ diff --git a/courses/algorithms/02-analysis/linear-vs-binary-search-worst-case.gif b/courses/algorithms/02-analysis/linear-vs-binary-search-worst-case.gif new file mode 100644 index 00000000..d2facc23 Binary files /dev/null and b/courses/algorithms/02-analysis/linear-vs-binary-search-worst-case.gif differ diff --git a/courses/algorithms/README.md b/courses/algorithms/README.md index c4d9bed9..234258e5 100644 --- a/courses/algorithms/README.md +++ b/courses/algorithms/README.md @@ -77,7 +77,7 @@ College dates for the Spring 2024 semester: --- - Week 2. 2024/01/22 - - Topic: Algorithm Analysis + - Topic: [Algorithm Analysis](02-analysis/README.md) - ### :material-code-array:{ .lg .middle } __Dynamic Data__ diff --git a/mkdocs.yml b/mkdocs.yml index b8b1d88c..35ff1a95 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -259,6 +259,8 @@ plugins: minify_html: true - exclude: glob: + - .git/* + - .github/* - node_modules/* - buildDocs/* - cmake-build-debug/*