|
1 | 1 | # [classes.wtf](https://classes.wtf/)
|
2 | 2 |
|
3 |
| -_A course catalog with full-text search that's so fast, you can't even perceive any delay._ |
| 3 | +_I just want to take a class about **\[X\]** but searching the online catalog is way too slow, and my results are irrelevant. WTF?_ |
4 | 4 |
|
5 |
| -Harvard's has many course search websites, but they are slow and tend to return poor quality results. This project is an attempt to take the problem seriously: write high-performance software and set good defaults so that people can get better, more salient suggestions, 100x faster. |
| 5 | + |
6 | 6 |
|
7 |
| -## WTF: How does it work? |
| 7 | +Harvard has many course search websites, but none of them are good. This project is an attempt to take the problem more seriously: write high-performance software and set great defaults so that people can get better, more useful suggestions, 100x faster. |
8 | 8 |
|
9 |
| -**Classes.wtf** implements a globally distributed custom search engine written in [Go](https://go.dev/), built on an in-memory [Redis](https://redis.io/) database that runs as a subprocess of the application. This supports full-text fuzzy and prefix search on all fields, along with a rich query syntax. |
| 9 | +## How does it work? |
10 | 10 |
|
11 |
| -The frontend is a static website built with [Svelte](https://svelte.dev/), and it sends search queries immediately after every keystroke. The goal is for the entire request, computation, response, and render pipeline to take under 30 milliseconds. |
| 11 | +**Classes.wtf** is a custom, distributed search engine written in [Go](https://go.dev/) that focuses on speed and quality of results. It's built on an in-memory [Redis](https://redis.io/) database that runs as a subprocess of the application. This index supports full-text fuzzy and prefix search on all fields, along with a rich query syntax. |
12 | 12 |
|
13 |
| -Observant readers will notice that the speed of light is not fast enough for data to travel around the world at this latency. This is okay though. We run multiple replicas at geographically distributed locations using [Fly.io](https://fly.io/) and route requests to the nearest one. Each replica runs its own full-text query engine, so they are completely independent. |
| 13 | +The frontend is a static website built with [Svelte](https://svelte.dev/), and it processes search queries immediately after every keystroke. The goal is for the entire {request, computation, response, and render} pipeline to take under 30 milliseconds. |
| 14 | + |
| 15 | +"Now hang on just a second," I hear you saying. The speed of light is not fast enough for data to travel around the world at this latency! But don't worry, this is fine. We run multiple replicas at geographically distributed locations using [Fly.io](https://fly.io/) and route requests to the nearest one. Each replica runs its own full-text query engine, so they are completely independent. |
14 | 16 |
|
15 | 17 | (The nearest server replica to Cambridge, MA lives in Secaucus, NJ, only 200 miles away.)
|
16 | 18 |
|
17 | 19 | ### FAQ
|
18 | 20 |
|
19 |
| -**Why did you make this?** I was frustrated by how annoying it was to search for classes. And I'm a systems software engineer, which pretty much gives me domain expertise in [making things faster](https://xkcd.com/1319/). |
| 21 | +**Why did you make this?** I was frustrated by how annoying it was to search for classes. And I'm a systems software engineer, which pretty much makes it my mandate to [make things faster](https://xkcd.com/1319/). |
20 | 22 |
|
21 |
| -**Why is it written in Go?** Because I wrote this in a weekend and needed a really fast systems language to iterate on while also having low latency. Go's simplicity and compile times helped with this. I'd probably rewrite it in Rust if I decided to spend a couple more weeks on it. |
| 23 | +**Why is it written in Go?** Because I wrote this in a weekend and needed a really fast systems language to iterate on while also having low latency. Go's simplicity and compile times helped with this. I might rewrite it in Rust if I decide to spend a couple more weeks on it. |
22 | 24 |
|
23 |
| -**Why are you using Redis?** It's really fast, it stores data in memory, the API is simple and robust, and it has a best-in-class full-text search module. For this size of dataset embedding Redis gives you unmatched performance, with a fraction of the cost and effort of alternatives. |
| 25 | +**Why are you using Redis?** It's really fast, it stores data in memory, the API is simple and robust, and it has a best-in-class full-text search module. For this size of dataset, embedding Redis gives you unmatched performance with a fraction of the cost and effort of alternatives. |
24 | 26 |
|
25 |
| -**Can you make this for my school? ** The code is all open-source, and you're welcome to take a look or port it! If you're doing this please also consider sending me a Twitter DM [@ekzhang1](https://twitter.com/ekzhang1) or an [email ](mailto:[email protected]), since I'd love to talk tech and make this bigger together. |
| 27 | +**Can you make this for my school? ** The code is all open-source, and you're welcome to take a look or port it! If you're doing this please also consider reaching out on Twitter [@ekzhang1](https://twitter.com/ekzhang1) or by [email ](mailto:[email protected]), since I'd love to talk tech and expand this into a shared course database together. |
26 | 28 |
|
27 | 29 | **Where is the data sourced?** The course catalog was indexed from publicly available course titles and descriptions online. See the code in the `datasource/` folder.
|
28 | 30 |
|
|
0 commit comments