|
| 1 | +# Staffjoy V2 |
| 2 | + |
| 3 | +[](https://travis-ci.org/Staffjoy/v2) [](https://moonlightwork.com/staffjoy) |
| 4 | + |
| 5 | +[Staffjoy is shutting down](https://blog.staffjoy.com/staffjoy-is-shutting-down-39f7b5d66ef6#.ldsdqb1kp), so we are open-sourcing our code. This the second version of our product, a ground-up rewrite intended for small businesses, like restaurants. You can learn about the design journey from V1 to V2 in [this blog post](https://blog.staffjoy.com/staffjoy-v2-ca15ff1a1169#.e7lmhde6v). |
| 6 | + |
| 7 | +We started building V2 in August 2016, became feature complete in November 2016, and [launched to the press in January 2017](http://venturebeat.com/2017/01/10/staffjoy-raises-1-2-million-to-help-small-businesses-manage-workflow-scheduling/). |
| 8 | + |
| 9 | +This is a *monorepo*, so all of the code for all of the services are in this repo. The core technologies are the [Bazel build system](https://bazel.build/), [Kubernetes](https://kubernetes.io/) (including its DNS for internal service discovery), [Redux](http://redux.js.org/), [Go](https://golang.org/), and [Yarn](https://yarnpkg.com/en/). In staging and production, we used [Google Container Engine](https://cloud.google.com/container-engine/) and their managed databases. |
| 10 | + |
| 11 | +## Credit |
| 12 | + |
| 13 | +The authors of the original code were [@philipithomas](https://github.com/philipithomas), [@samdturner](https://github.com/samdturner), [@andhess](https://github.com/andhess), and some contractors. This is a fork of the internal repository. For security purposes, the Git history has been squashed. |
| 14 | + |
| 15 | +## Services |
| 16 | + |
| 17 | +[Read about the V2 architecture on our blog](https://blog.staffjoy.com/staffjoys-v2-architecture-9d2fcb4015fd#.pggmlbtmw) |
| 18 | + |
| 19 | +[](https://blog.staffjoy.com/staffjoys-v2-architecture-9d2fcb4015fd#.pggmlbtmw) |
| 20 | + |
| 21 | +* Faraday proxies all traffic from external services to internal ones. It also tells backend services whether a user is logged in. It's the only service that has a public IP address! |
| 22 | +* www (www.staffjoy.com) is the main marketing website. It also handles login and logout. |
| 23 | +* myaccount (myaccount.staffjoy.com) is a single-page javascript app that lets users modify their accounts |
| 24 | +* account-gateway (account.staffjoy.com) is the externally-available REST api for modifying accounts. It converts REST to gRPC for the accounts-datastore |
| 25 | +* accounts-server is the internal system that processes gRPC calls and stores/retrieves information using the account database. |
| 26 | +* company-gateway (company.staffjoy.com) is the externally-available REST api for modifying companies. It converts REST to gRPC for the companys-datastore |
| 27 | +* company-server is the internal system that processes gRPC calls and stores/retrieves information using the company database. |
| 28 | +* whoami (whoami.staffjoy.com) is a website that sends information about the current web session for easy access in the front-end. |
| 29 | +* superpowers (superpowers.staffjoy-v2.local) is a development-only website that lets you gain super user powers across Staffjoy (denoted as "support" flag on user accounts) |
| 30 | +* ical (ical.staffjoy-v2.local) is service serving up a worker's shift list through ical |
| 31 | + |
| 32 | +### External API Standards |
| 33 | + |
| 34 | +* Services should be RESTful JSON over HTTPS |
| 35 | +* Serve the spec at `/swagger.json` |
| 36 | +* Use the `apidocs` package to serve a swagger UI at `/ui/` |
| 37 | + |
| 38 | +## Dev |
| 39 | + |
| 40 | +### Getting started |
| 41 | + |
| 42 | +Welcome to Staffjoy! |
| 43 | + |
| 44 | +We use a **monorepo** that stores all of our code in this single repo. We use Vagrant to run a kubernetes cluster locally on your laptop. This makes it easy to run all of Staffjoy's services. |
| 45 | + |
| 46 | +### Setting up your Gopath |
| 47 | + |
| 48 | +If you are running Go code, you should [set up your $GOPATH](https://golang.org/doc/install), then clone this repository into the `v2.staffjoy.com` package: |
| 49 | + |
| 50 | +``` |
| 51 | +mkdir -p $GOPATH/src/ |
| 52 | +git clone [email protected]:Staffjoy/staffjoy.git $GOPATH/src/v2.staffjoy.com/ |
| 53 | +``` |
| 54 | + |
| 55 | +### One-time dependencies on host machine (laptop) |
| 56 | + |
| 57 | +* [Virtualbox](https://www.virtualbox.org/) |
| 58 | +* [Vagrant](https://www.vagrantup.com/docs/getting-started/), then run `vagrant up` to boot the dev server. |
| 59 | +* Vagrant host manager: `vagrant plugin install vagrant-hostmanager` |
| 60 | +* unison (see below) |
| 61 | +* [modd](https://github.com/cortesi/modd) (either from Go source or binary available [here]) |
| 62 | + |
| 63 | +### Installing Unison |
| 64 | + |
| 65 | +Unison syncs files between the host and the VM in a way that preserves simlinks. |
| 66 | + |
| 67 | +Unfortunately, installing it on OSX is a bit of a pain because the version and the compiler must be the same as the one in Vagrant! |
| 68 | + |
| 69 | +First, install OCaml version 3.12 - you may need to say "yes" to some command prompts: |
| 70 | + |
| 71 | +``` |
| 72 | +wget https://raw.github.com/ocaml/opam/master/shell/opam_installer.sh -O - | sh -s /usr/local/bin 3.12.1 |
| 73 | +``` |
| 74 | + |
| 75 | +Check that `ocaml -version` is 3.12.1. If it is not, try removing `/usr/local/bin/ocaml` and running `opam switch 3.12.1`. |
| 76 | + |
| 77 | +Then, compile unison with this version of ocaml. You may be prompted for your password. |
| 78 | + |
| 79 | +``` |
| 80 | +cd /tmp |
| 81 | +curl -O https://www.seas.upenn.edu/~bcpierce/unison/download/releases/unison-2.40.102/unison-2.40.102.tar.gz |
| 82 | +tar -xvzf unison-2.40.102.tar.gz |
| 83 | +cd unison-2.40.102 |
| 84 | +make UISTYLE=text |
| 85 | +sudo cp unison /usr/local/bin |
| 86 | +``` |
| 87 | + |
| 88 | + |
| 89 | +### Running the Environment |
| 90 | + |
| 91 | + |
| 92 | +Run `make dev`. Code will boot and run at [staffjoy-v2.local](http://www.staffjoy-v2.local). Note that the first time you do this could take up to 45 minutes in order to provision the VM! |
| 93 | + |
| 94 | +Changes will trigger an automatic rebuild and redeployment. (Check deployment progress at [kubernetes.staffjoy-v2.local/ui/](http://kubernetes.staffjoy-v2.local/ui/)). End the dev server with `control + c` (and it will automatically shut off the VM). |
| 95 | + |
| 96 | +**Known Bug on OSX Sierra**: If Vagrant cannot find download the box, run `sudo rm /opt/vagrant/embedded/bin/curl`. ([Bug tracking link](https://github.com/Varying-Vagrant-Vagrants/VVV/issues/354)) |
| 97 | + |
| 98 | +If you run into issues with stuck deployments in development - then run `make dev-k8s-fix` then re-run `make dev`. |
| 99 | + |
| 100 | +### Accessing the environment |
| 101 | + |
| 102 | +Access the VM by running `vagrant ssh`. Code is located in `/home/vagrant/golang/src/v2.staffjoy.com/` (aliased to `$STAFFJOY`, i.e. `cd $STAFFJOY`). |
| 103 | + |
| 104 | +To build code and run it locally, in vagrant go to the code directory `$STAFFJOY` in vagrant, then run `make dev-build` for a one-time build. |
| 105 | + |
| 106 | +If things are really goofing, run `vagrant destroy -f` then rebuild. |
| 107 | + |
| 108 | +## Debugging |
| 109 | + |
| 110 | +* There is a **known bug** where kubernetes does not come back after vagrant halts. To identify this, ssh into vagrant (`vagrant ssh`), then examine running docker containers (`docker ps`). If it's not running dozens of contianers, there's a problem. **To fix this:**, run `make dev-k8s-fix`. (Your local data may be wiped out). |
| 111 | +* If files are not syncing between your laptop and vagrant (to Vagrant's `$STAFFJOY` directory) - try running `make dev` again, or manually running unison (the syncer) on the host machine with `./vagrant/unison.sh`. |
| 112 | +* If your machine can't keep up with autobuilding on changes, it may be preferred to stop using `make dev` and to instead call its two subcommands separately (manually): `./vagrant/unison.sh` to sync files on your computer, then SSHing into Vagrant and manually triggering builds (`cd $STAFFJOY && make dev-build`) |
| 113 | + |
| 114 | +### Development resources |
| 115 | + |
| 116 | +* [Kubernetes UI](http://kubernetes.staffjoy-v2.local/ui/). We use the `development` namespace. You can see logs from a "pod" (container) through the UI ([link](http://kubernetes.staffjoy-v2.local/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard/#/pod?namespace=development)) |
| 117 | +* [HTTP Debugger (for faraday)](http://faraday.staffjoy-v2.local/) - use this to examine headers being sent to internal systems |
| 118 | +* [Superpowers lets you do magic things](http://superpowers.staffjoy-v2.local) - use it while logged in to get `support` api permissions in dev. |
| 119 | + |
| 120 | +## Go |
| 121 | + |
| 122 | +### Assets in Go |
| 123 | + |
| 124 | +If you are loading assets like templates, CSS, JS, etc - you need to package the the data into the binary. Otherwise, the app will ship and it won't be able to find the assets! |
| 125 | + |
| 126 | +To do this, use the [go-bindata](https://github.com/jteeuwen/go-bindata) project. If you modify any of the asset files, you will need to rebuild them then commit the resulting `bindata.go` file and commit it. You have been warned! |
| 127 | + |
| 128 | +Most services provide a `build.sh` file that compiles all the data that needs to be committed. |
| 129 | + |
| 130 | +### Development tools |
| 131 | + |
| 132 | +The tool [GoConvey](https://github.com/smartystreets/goconvey) is great for seeing tests. |
| 133 | + |
| 134 | +### Environment variables for configuration |
| 135 | + |
| 136 | +* `ENV`: Set to `development`,`staging`, or `production`. Null defaults to `development` |
| 137 | +* `SENTRY_DSN`: Set to the [Sentry](https://getsentry.com) api key in every Go service for proper error tracking and reporting |
| 138 | + |
| 139 | +## Protocol Buffers |
| 140 | + |
| 141 | +If you modify the files in `protobuf/`, run `make protobuf` to recompile all of the generated files. |
| 142 | + |
| 143 | +If you're getting started with protocol buffers, here are some resources: |
| 144 | + |
| 145 | +* [Protocol Buffers](https://developers.google.com/protocol-buffers/) |
| 146 | +* [gRPC](http://grpc.io) |
| 147 | +* [gRPC health checks](https://github.com/grpc/grpc/blob/master/doc/health-checking.md) |
| 148 | + |
| 149 | +## Working Offline |
| 150 | + |
| 151 | +* Email will break, but you can look at the system logs for the email service to see what would have been sent. (Useful for grabbing account activation links!) |
| 152 | +* See all Go documentation installed on the host machie with `godoc -http=":8080"`. You'll be able to see all docs at [localhost:8080](http://localhost:8080/) |
0 commit comments