fittin.gs technology summary

11 Dec 2019

I started playing Eve again recently. It’s a game where you build spaceships by fitting equipment to them, so we use the word fit or fitting when building a ship. Then you try to blow other spaceships up without blowing up yours. It’s really hard when you start to know how to fit your spaceship. There’s a site that lists most blowups in the game. While you can use it to try to figure out fits, that site isn’t quite geared toward that purpose, so searching it can be slow. I made a site that scrapes it and puts the data into a more searchable and digestable format.

It is called fittin.gs, and I’d like to describe how I built it.

I used some technology I’m familiar with (Netlify, Go, CockroachDB, React) and learned some new tech (GitHub Actions, Cockroach Cloud, React hooks, Typescript, Google Cloud Run) while building this site. I’m going to give a brief description of each and show how they are used. You can play along at home with the code.

Goals

My goals when making a website are:

React

The frontend was created using react-create-app which is a great helper script to sketch out a working react app with live reload on save. React is a javascript library for building frontends where you write HTML in your JS. Each component and page can be its own function. Mutable and non-mutable state changes allow it to know what to update when users click on things and actions need to be taken like requests to the backend.

React recently added hooks and I liked them. They simplify somewhat the use of react internals. A few gotchas but probably because I skim the docs instead of read them thoroughly, and so I miss lots of things.

Typescript

Typescript is thing that allows you to type-annotate your javascript and it complains on errors. It is aware of the HTML-in-JS stuff of react. It adds all of the normal type stuff we like about typed languages. This was my first time using it and I felt very safe when I refactored internal stuff. Definitely saved me some bugs.

materials.style

materials.style is a CSS file that allows you to use CSS variables for colors that also respects the dark/light setting of your browser. It’s a tiny little thing I made a while ago to solve this problem. I don’t love it because link colors are always weird in material design and I don’t do the full button and component stuff they specify, and as a result things look weird sometimes (my fault, not theirs).

Netlify

Netlify is a CDN that can build your site from a github repo and execute arbitrary stuff during build. On push to github master, they pull the repo, build the frontend, and deploy it to their CDN.

Go

Go is my backend language of choice. It is great for making websites and I understand how it works. It easily builds into a tiny docker image for simple deployment.

I used modd to watch for .go file changes and reload the backend on save.

CockroachDB

CockroachDB is a distributed SQL database and my employer. It acts similar to Postgres but is very good at distributed replication while maintaining data consistency.

CockroachCloud was recently announced, which is a service where Cockroach Labs runs a cluster for you and hands you a connection string in exchange for a credit card. I’ve been using it for a few weeks now and overall I’m pleased by how much stuff I didn’t have to care about. It’s just running somewhere and I don’t need to manage it. Great.

Google Cloud Run

Cloud Run is a thing where you give Google a docker image and it gives you a URL. When HTTP requests are made to that URL, they spin up the docker image and route the request to it. More requests spin up more images. You pay only for the time a web request is being served by your image. It scales down to zero instances when no requests are incoming. So kinda like app engine but with docker images. Yes finally.

The Go backend is built into this docker image. The frontend served by netlify makes API requests to this cloud run URL. The cloud run backend then connects to the CockroachCloud server. So the whole request chain is like netlify -> cloud run -> cockroach cloud (but obviously the browser is actually making the request to cloud run, not netlify). Cloud run and CockroachCloud are in the same GCE region though, so things stay fast.

They also have a Cloud Scheduler service that is like cron for HTTP. I use it to hit the “sync data from the source site” route every few minutes. That route processes as much unprocessed data as there is and then shuts down so the cloud run instances are billed for as little time as possible.

GitHub Actions

Actions is GitHub’s CI/CD stuff. You can have github test on PRs and deploy on push to master, or whatever you want. When I push to master, github starts the docker image build and then tells cloud run about it.

One super important thing you need for these kinds of deploy-on-push things is a secret store for your DB connection strings, certs, and passwords. These are integrated into actions such that your action deploy code can use them, but the secrets are otherwise invisible. This makes it possible to safely have my site and deploy scripts be open source, and still have safe secrets.

Conclusion

Oh hell there are a lot of things you need to know to develop and deploy a quality website. And I’m missing a bunch like database migrations. There’s just an unreasonable amount of crap you have to learn to be a successful web developer. I really hope ideas from dark utterly restructure how we build websites.

Yet, I’m really happy where this site is now. I’m going to add more features to it that only Eve people care about. But from a tech perspective this is the template I will copy when I do my next site.

See you in space. o7.