Separate tracker build + tests w/ Vite, Turborepo, & Playwright (#132)

* First start at turborepo

* Remove temp turbo files, fix public dir

* Fix gitignore

* Add tracker subrepo

* Move typescript to root package

* Move prettier to root project

* Empty tsconfig

* Copy dist/tracker.js to server /public dir

* Fix tests, precommit hooks

* Add deploy task

* Run single turbo run command on precommit

* Add playwright tests for tracker script

* Add test to verify request to /collect

* Remove example tests

* Test asserts query string is good

* Remove tmp files

* Attempt to fix GH actions

* GH action fix attempt #2

* Add wrangler to root package.json

* Update package-lock

* Add "preview" turbo command, edit README w/ new turbo commands

* Convert tracker to typescript

* Get tracker tests into folders (get ready for multiple tests)

* Rename tracker.js to reporter.js

* Add missing /reporter dir

* Add data-site-id initialization method

* Update README for simplified initialization method

* Rename workspaces to packages

* Rename test file to better mirror 01_legacyInit

* Add eslint to reporter package

* Typecheck needs to build first

* Rollback newly introduced API methods (unnecessary)

* Update typescript

* Same typescript version in both packages

* Update package lock

* Rename reporter back to tracker

* Update eslint typescript tooling to suppress warnings

* Update turbo dependencies

Author: benvinegar

.github/workflows/cd.yaml

@@ -14,7 +14,7 @@ jobs:
               with:
                   node-version-file: "package.json"
             - run: npm install
-            - run: npm run build
+            - run: npx turbo build
             - uses: cloudflare/wrangler-action@v3
               with:
                   apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}

.github/workflows/ci.yaml

@@ -13,10 +13,7 @@ jobs:
               with:
                   node-version-file: "package.json"
             - run: npm install
-            - run: npm run lint
-            - run: npm run build
-            - run: npm run typecheck
-            - run: npm run test-ci
+            - run: npx turbo lint build typecheck test-ci --concurrency=1
             - name: Upload coverage reports to Codecov
               uses: codecov/codecov-action@v3
               env:

.gitignore

@@ -1,11 +1,13 @@
 node_modules
 
-/.cache
-/.wrangler
-/build
-/public/build
+.cache
+.wrangler
+build
+dist
+public/build
 .env
 
 .dev.vars
 
-coverage
+coverage
+.turbo

.husky/pre-commit

@@ -11,8 +11,6 @@ echo "$FILES" | xargs ./node_modules/.bin/prettier --ignore-unknown --write
 # Add back the modified/prettified files to staging
 echo "$FILES" | xargs git add
 
-npm test
-npm run lint
-npm run typecheck
+npx turbo test lint typecheck --concurrency=1
 
 exit 0

README.md

@@ -22,52 +22,42 @@ If you don't have one already, [create a Cloudflare account here](https://dash.c
 1. Enable [Cloudflare Analytics Engine beta](https://developers.cloudflare.com/analytics/analytics-engine/get-started/) for your account ([screenshot](https://github.com/benvinegar/counterscale/assets/4562878/ad1b5712-2344-4489-a684-685b876635d1))
     1. If this is your first time using Workers, you have to create a Worker before you can enable the Analytics Engine. Navigate to Workers & Pages > Overview, click the "Create Worker" button ([screenshot](./docs/create-worker.png)) to create a "Hello World" worker (it doesn't matter what you name this Worker as you can delete it later).
 1. Create a [Cloudflare API token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/). This token needs `Account.Account Analytics` permissions at a minimum ([screenshot](./docs/api-token.png)).
-   * _WARNING: Keep this window open or copy your API token somewhere safe (e.g. a password manager), because if you close this window you will not be able to access this API token again and have to start over._
+    - _WARNING: Keep this window open or copy your API token somewhere safe (e.g. a password manager), because if you close this window you will not be able to access this API token again and have to start over._
 
 ### Deploy Counterscale
 
 1. Download the [latest Counterscale release](https://github.com/benvinegar/counterscale/releases/latest) (or clone the repository) and extract the source files to a folder.
 1. With your terminal, navigate to the folder containing the source files.
 1. Run `npm install`
 1. Run `npx wrangler pages project create counterscale` and create a new Pages project.
-    1. You will be prompted to enter the "production branch name". Just use the default provided (e.g. "main" or "production").
-    * _NOTE: If this is your first time invoking `wrangler` on the terminal, you will be prompted to sign into your Cloudflare account._
+    1. You will be prompted to enter the "production branch name". Just use the default provided.
+    - _NOTE: If this is your first time invoking `wrangler` on the terminal, you will be prompted to sign into your Cloudflare account._
 1. Run `npx wrangler pages secret put CF_BEARER_TOKEN` → when prompted, paste the API token you created
 1. Run `npx wrangler pages secret put CF_ACCOUNT_ID` → when prompted, paste your Cloudflare Account ID
-    * Find your account ID by visiting Workers and Pages > Overview. It is displayed on the right hand side of the screen.
-1. Run `npm run deploy` – this will do several things:
+    - Find your account ID by visiting Workers and Pages > Overview. It is displayed on the right hand side of the screen.
+1. Run `npx turbo deploy` – this will do several things:
     1. Create a new Analytics Engine dataset, called `metricsDataset`
     1. Deploy the site and give you the deployment URL.
 1. The site should now be deployed. Visit `https://{subdomain-emitted-from-npm-run-deploy}.pages.dev`.
-    * NOTE: _It may take take a few minutes before the subdomain becomes live._
+    - NOTE: _It may take take a few minutes before the subdomain becomes live._
 
-### Install the Tracker Script on Your Website(s)
+### Install the Tracking Script on Your Website(s)
 
 When Counterscale is deployed, it makes `tracker.js` available at the URL you deployed to:
 
 ```
 https://{subdomain-emitted-from-npm-run-deploy}.pages.dev/tracker.js
 ```
 
-To start tracking website traffic on your web property, copy/paste the following snippet into your website HTML:
+To start reporting website traffic from your web property, copy/paste the following snippet into your website HTML:
 
 ```html
-<script>
-    (function () {
-        window.counterscale = {
-            q: [["set", "siteId", "your-unique-site-id"], ["trackPageview"]],
-        };
-    })();
-</script>
-<script
-    id="counterscale-script"
+<script 
     src="https://{subdomain-emitted-from-npm-run-deploy}.pages.dev/tracker.js"
     defer
 ></script>
 ```
 
-Be sure to replace `your-unique-site-id` with a unique string/slug representing your web property. Use a unique site ID for each property you place the tracking script on.
-
 ## Troubleshooting
 
 If the website is not immediately available (e.g. "Secure Connection Failed"), it could be because Cloudflare has not yet activated your subdomain (yoursubdomain.workers.dev). This process can take a minute; you can check in on the progress by visiting the newly created worker in your Cloudflare dashboard (Workers & Pages → counterscale).
@@ -80,16 +70,16 @@ The deployment URL can always be changed to go behind a custom domain you own. [
 
 ### Config
 
-To get started, in the project root, copy `.dev.vars.example` to `.dev.vars`.
+To get started, in the `packages/server` folder, copy `.dev.vars.example` to `.dev.vars`.
 
 Open `.dev.vars` and enter the same values for `CF_BEARER_TOKEN` and `CF_ACCOUNT_ID` you used earlier.
 
 ### Running the Server
 
 Counterscale is built on Remix and Cloudflare Workers. In development, you have two options:
 
-1. `npm run dev` → This runs the Vite development server in Node.js. This server will automatically rebuild files when you change them, but it does not best reflect Cloudflare's serverless platform.
-2. `npm run preview` → This runs Cloudflare's Miniflare server with a build of the Remix files. This closer matches the deployment environment, but does not (yet) automatically rebuild your app.
+1. `npx turbo dev` → This runs the Vite development server in Node.js. This server will automatically rebuild files when you change them, but it does not best reflect Cloudflare's serverless platform.
+2. `npx turbo preview` → This runs Cloudflare's Miniflare server with a build of the Remix files. This closer matches the deployment environment, but does not (yet) automatically rebuild your app.
 
 ## Notes
 
@@ -99,8 +89,8 @@ There is only one "database": the Cloudflare Analytics Engine dataset, which is
 
 Right now there is no local "test" database. This means in local development:
 
--   Writes will no-op (no hits will be recorded)
--   Reads will be read from the production Analaytics Engine dataset (local development shows production data)
+- Writes will no-op (no hits will be recorded)
+- Reads will be read from the production Analaytics Engine dataset (local development shows production data)
 
 ### Sampling
 
@@ -116,10 +106,10 @@ The primary goal of Counterscale is to be super easy to self-host and maintain.
 
 To achieve that:
 
--   There should be no application state outside of CF Analytics Engine
-    -   e.g. no additional relational database like MySQL, PostgreSQL, etc.
-    -   That means no `users` table, no `sites` table, etc.
-    -   This also means retention will be limited by what CF Analytics Engine provides. While it could be possible to stand up a "hit counter" for long-lived data (e.g. years), that would mean another database, which we will not pursue.
--   We prioritize backwards compatibility
-    -   New `metricsDataset` columns can be added, but old columns cannot be removed or renamed (they can however, be "forgotten").
-    -   That also means it's okay if a feature only works during a period where the data is active.
+- There should be no application state outside of CF Analytics Engine
+    - e.g. no additional relational database like MySQL, PostgreSQL, etc.
+    - That means no `users` table, no `sites` table, etc.
+    - This also means retention will be limited by what CF Analytics Engine provides. While it could be possible to stand up a "hit counter" for long-lived data (e.g. years), that would mean another database, which we will not pursue.
+- We prioritize backwards compatibility
+    - New `metricsDataset` columns can be added, but old columns cannot be removed or renamed (they can however, be "forgotten").
+    - That also means it's okay if a feature only works during a period where the data is active.