Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

license, readme, minor updates #75

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 11 additions & 71 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,85 +1,25 @@
# Contributing

Given external users will not have write to the branches in this repository, you'll need to follow the forking process to open a PR - [here](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork) is a guide from github on how to do so.

Please also read our main [contributing guide](https://github.com/theopenlane/.github/blob/main/CONTRIBUTING.md) in addition to this one; the main guide mostly says that we'd like for you to open an issue first but it's not hard-required, and that we accept all forms of proposed changes given the state of this code base (in it's infancy, still!)

## Pre-requisites to a PR

This repository contains a number of code generating functions / utilities which take schema modifications and scaffold out resolvers, graphql API schemas, openAPI specifications, among other things. To ensure you've generated all the necessary dependencies run `task pr`; this will run the entirety of the commands required to safely generate a PR. If for some reason one of the commands fails / encounters an error, you will need to debug the individual steps. It should be decently easy to follow the `Taskfile` in the root of this repository.

### Pre-Commit Hooks

We have several `pre-commit` hooks that should be run before pushing a commit. Make sure this is installed:

```bash
brew install pre-commit
pre-commit install
```

You can optionally run against all files:

```bash
pre-commit run --all-files
```

## Starting the Server

1. Copy the config, this is in .gitignore so you do not have to worry about accidentally committing secrets
Please read the [contributing](.github/CONTRIBUTING.md) guide as well as the [Developer Certificate of Origin](https://developercertificate.org/). You will be required to sign all commits to the Openlane project, so if you're unfamiliar with how to set that up, see [github's documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification).

```bash
cp ./config/config-dev.example.yaml ./config/.config.yaml
```

1. Update the configuration with whatever respective settings you desire; the defaults inside should allow you to run the server without a problem

1. Use the task commands to start the server

Run the core server in development mode with dependencies in docker

```bash
task run-dev
```

Run fully in docker

```bash
task docker:all:up
```

1. In a separate terminal, with the server running, you can create a verified test user by running:

```bash
task cli:user:all
```

1. Once this command has finished ^, you can login and perform actions as user `[email protected]` with password `mattisthebest1234!`

## Creating Queries in GraphQL
Given external users will not have write to the branches in this repository, you'll need to follow the forking process to open a PR - [here](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork) is a guide from github on how to do so.

The best method of forming / testing queries against the server is to run `task docker:rover` which will launch an interactive query UI.
## Licensing

If you are running the queries against your local repo, you will have CORS issues using the local running apollo. Instead, its recommended to use the [apollo sandbox](https://studio.apollographql.com/sandbox/explorer) and ensure the following origin is allowed in your `config/.config.yaml`
This repository contains open source software that comprises the Openlane stack which is open source software under [Apache 2.0](LICENSE). Openlane's SaaS / Cloud Services are products produced from this open source software exclusively by theopenlane, Inc. This product is produced under our published commercial terms (which are subject to change). Any logos or trademarks in our repositories in [theopenlane](https://github.com/theopenlane) organization are not covered under the Apache License and are trademarks of theopenlane, Inc.

```
server:
cors:
allowOrigins:
- https://studio.apollographql.com
```
Others are allowed to make their own distribution of this software or include this software in other commercial offerings, but cannot use any of the Openlane logos, trademarks, cloud services, etc.

## OpenFGA Playground
## Security

You can load up a local openFGA environment with the compose setup in this repository; `task fga:up` - this will launch an interactive playground where you can model permissions model(s) or changes to the models
We take the security of our software products and services seriously, including our commercial services and all of the open source code repositories managed through our Github Organizations, such as [theopenlane](https://github.com/theopenlane). If you believe you have found a security vulnerability in any of our repositories or in our SaaS offering(s), please report it to us through coordinated disclosure.

## Creating a new Schema
**Please do NOT report security vulnerabilities through public github issues, discussions, or pull requests!**

To ease the effort required to add additional schemas into the system a template + task function has been created. This isn't doing anything terribly complex, but it's attempting to ensure you have the _minimum_ set of required things needed to create a schema - most notably: you need to ensure the IDMixin is present (otherwise you will get ID type conflicts) and a standard set of schema annotations.
Instead, please send an email to `[email protected]` with as much information as possible to best help us understand and resolve the issues. See the security policy attached to this repository for more details.

NOTE: you still have to make intelligent decisions around things like the presence / integration of hooks, interceptors, policies, etc. This is saving you about 10 seconds of copy-paste, so don't over estimate the automation, here.
## Questions?

To generate a new schema, you can run `task newschema -- [yourschemaname]` where you replace the name within `[]`. Please be sure to note that this isn't a command line flag so there's a space between `--` and the name.
You can email us at `[email protected]`, open a github issue in this repository, or reach out to [matoszz](https://github.com/matoszz) directly.

### Migrations

We use [atlas](https://atlasgo.io/) and [goose](https://github.com/pressly/goose) to create and manage our DB migrations - you can trigger one via `task atlas:create` and that will generate the necessary migrations. There should be a new migration file created in `db/migrations`, `db/migrations-goose-postgre` and `db/migrations-goose-sqlite`. On every PR, the Atlas integration also creates comments with any issues related to the schema changes / migrations.
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ repos:
hooks:
- id: yamlfmt
- repo: https://github.com/crate-ci/typos
rev: v1.24.5
rev: v1.24.6
hooks:
- id: typos
- repo: local
Expand Down
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@
same "printed page" as the copyright notice for easier
identification within third-party archives.

Copyright 2024 The Open Lane, Inc.
Copyright 2024 theopenlane, Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand Down
97 changes: 59 additions & 38 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
[![Build status](https://badge.buildkite.com/a3a38b934ca2bb7fc771e19bc5a986a1452fa2962e4e1c63bf.svg?branch=main)](https://buildkite.com/theopenlane/core)
[![Go Reference](https://pkg.go.dev/badge/github.com/theopenlane/core.svg)](https://pkg.go.dev/github.com/theopenlane/core)
[![License: Apache 2.0](https://img.shields.io/badge/License-Apache2.0-brightgreen.svg)](https://opensource.org/licenses/Apache-2.0)
[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=theopenlane_core&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=theopenlane_core)

</div>

Expand All @@ -22,6 +23,7 @@ At it's core, this repo is a collection of services built on top of an entity fr
- Code generated audit / history tables for defined schemas

On top of this powerful core we also have an incredible amount of pluggable, extensible services:

- Authentication: we today support password, OAuth2 / Social login providers (Github, Google), Passkeys as well as standard OIDC Discovery flows
- Multi-factor: built-in 2FA mechanisms, TOTP
- Authorization: extensible and flexible permissions constructs via openFGA based on Google Zanzibar
Expand All @@ -34,31 +36,7 @@ On top of this powerful core we also have an incredible amount of pluggable, ext

## Development

Developing against this repo involves a few mandatory tools; please read up on these and familiarize yourself if you're interested in making additions or changes!

1. [ent](https://entgo.io/) - insane entity mapping tool, definitely not an ORM but kind of an ORM (handles our relational data storage, mappings, codegen processes)
1. [atlas](https://atlasgo.io/) - Schema generation and migrations (can be disabled in lieu of migrations on disk)
1. [goose](https://github.com/pressly/goose) - Secondary database migration utility we also use for seeding data
1. [gqlgen](https://gqlgen.com/) - Code generation + GraphQL server building from from `ent` schema definitions
1. [gqlgenc](https://github.com/Yamashou/gqlgenc) - Client building utilities with GraphQL
1. [openfga](https://openfga.dev/) - Flexible authorization/permission engine inspired by Google Zanzibar
1. [echo](https://echo.labstack.com/) - High performance, extensible, minimalist Go web framework
1. [koanf](https://github.com/knadh/koanf) - Configuration management library which parses command line arguments, Go structs + creates our main configuration files

We also leverage many secondary technologies in use, including (but not limited to!):

1. [taskfile](https://taskfile.dev/usage/) - So much better than Make zomg
1. [redis](https://redis.io/) - in-memory datastore used for sessions, caching
1. databases:
* [postgres](https://www.postgresql.org/)
* [libsql](https://turso.tech/libsql)
* [sqlite](https://www.sqlite.org/)
1. [golangci-lint](https://github.com/golangci/golangci-lint) - an annoyingly opinionated linter
1. [buildkite](https://buildkite.com/theopenlane) - our CI system of choice (with github actions providing some intermediary support)

All of these components are bundled into our respective Docker images; for additional information / instructions, see the [contributing guide](.github/CONTRIBUTING.md) in this repository. We're constantly adding and changing things, but have tried to list all the great open source tools and projects we rely on; if you see your project (or one you use) in here and wish to list it, feel free to open a PR!

## Dependencies
### Dependencies

The vast majority of behaviors of the system can be turned on or off by updating the configuration parameters found in `config`; in some instances, we've made features or integrations with third party systems which are "always on", but we're happy to receive PR's wrapping those dependencies if you are interested in running the software without them!

Expand All @@ -70,28 +48,71 @@ Setup [Taskfile](https://taskfile.dev/installation/) by following the instructio

See the [README](/config/README.md) in the `config` directory.

## Deploying
### Starting the Server

The only "supported" method of deploying today is locally, but we have a WIP Helm chart which can be found [here](https://github.com/theopenlane/helm-charts)
1. Copy the config, this is in .gitignore so you do not have to worry about accidentally committing secrets

## Contributing
```bash
cp ./config/config-dev.example.yaml ./config/.config.yaml
```

1. Update the configuration with whatever respective settings you desire; the defaults inside should allow you to run the server without a problem

1. Use the task commands to start the server

Run the core server in development mode with dependencies in docker

```bash
task run-dev
```

Run fully in docker

Please read the [contributing](.github/CONTRIBUTING.md) guide as well as the [Developer Certificate of Origin](https://developercertificate.org/). You will be required to sign all commits to the [theopenlane](https://github.com/theopenlane) organization, so if you're unfamiliar with how to set that up, see [github's documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification).
```bash
task docker:all:up
```

## Licensing
1. In a separate terminal, with the server running, you can create a verified test user by running:

This repository contains `core` which is open source software under [Apache 2.0](LICENSE). Openlane is a product produced from this open source software exclusively by The Open Lane, Inc. This product is produced under our published commercial terms (which are subject to change), and any logos or trademarks in this repository or the broader [theopenlane](https://github.com/theopenlane) organization are not covered under the Apache License.
```bash
task cli:user:all
```

Others are allowed to make their own distribution of this software or include this software in other commercial offerings, but cannot use any of the Openlane logos, trademarks, cloud services, etc.
1. Once this command has finished ^, you can login and perform actions as user `[email protected]` with password `mattisthebest1234`

## Security
### Creating Queries in GraphQL

We take the security of our software products and services seriously, including all of the open source code repositories managed through our Github Organizations, such as [theopenlane](https://github.com/theopenlane). If you believe you have found a security vulnerability in any of our repositories, please report it to us through coordinated disclosure.
The best method of forming / testing queries against the server is to run `task docker:rover` which will launch an interactive query UI.

**Please do NOT report security vulnerabilities through public github issues, discussions, or pull requests!**
If you are running the queries against your local repo, you will have CORS issues using the local running apollo. Instead, its recommended to use the [apollo sandbox](https://studio.apollographql.com/sandbox/explorer) and ensure the following origin is allowed in your `config/.config.yaml`

Instead, please send an email to `[email protected]` with as much information as possible to best help us understand and resolve the issues. See the security policy attached to this repository for more details.
```
server:
cors:
allowOrigins:
- https://studio.apollographql.com
```

## Questions?
### OpenFGA Playground

You can load up a local openFGA environment with the compose setup in this repository; `task fga:up` - this will launch an interactive playground where you can model permissions model(s) or changes to the models

### Creating a new Schema

To ease the effort required to add additional schemas into the system a template + task function has been created. This isn't doing anything terribly complex, but it's attempting to ensure you have the _minimum_ set of required things needed to create a schema - most notably: you need to ensure the IDMixin is present (otherwise you will get ID type conflicts) and a standard set of schema annotations.

NOTE: you still have to make intelligent decisions around things like the presence / integration of hooks, interceptors, policies, etc. This is saving you about 10 seconds of copy-paste, so don't over estimate the automation, here.

To generate a new schema, you can run `task newschema -- [yourschemaname]` where you replace the name within `[]`. Please be sure to note that this isn't a command line flag so there's a space between `--` and the name.

### Migrations

We use [atlas](https://atlasgo.io/) and [goose](https://github.com/pressly/goose) to create and manage our DB migrations - you can trigger one via `task atlas:create` and that will generate the necessary migrations. There should be a new migration file created in `db/migrations`, `db/migrations-goose-postgre` and `db/migrations-goose-sqlite`. On every PR, the Atlas integration also creates comments with any issues related to the schema changes / migrations.

## Deploying

The only "supported" method of deploying today is locally, but we have a WIP Helm chart which can be found [here](https://github.com/theopenlane/helm-charts)

## Contributing

You can email us at `[email protected]`, open a github issue in this repository, or reach out to [matoszz](https://github.com/matoszz) directly.
See the [contributing](.github/CONTRIBUTING.md) guide for more information
25 changes: 25 additions & 0 deletions docs/tools.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Tools

Developing against this repo involves a few mandatory tools; please read up on these and familiarize yourself if you're interested in making additions or changes!

1. [ent](https://entgo.io/) - insane entity mapping tool, definitely not an ORM but kind of an ORM (handles our relational data storage, mappings, codegen processes)
1. [atlas](https://atlasgo.io/) - Schema generation and migrations (can be disabled in lieu of migrations on disk)
1. [goose](https://github.com/pressly/goose) - Secondary database migration utility we also use for seeding data
1. [gqlgen](https://gqlgen.com/) - Code generation + GraphQL server building from from `ent` schema definitions
1. [gqlgenc](https://github.com/Yamashou/gqlgenc) - Client building utilities with GraphQL
1. [openfga](https://openfga.dev/) - Flexible authorization/permission engine inspired by Google Zanzibar
1. [echo](https://echo.labstack.com/) - High performance, extensible, minimalist Go web framework
1. [koanf](https://github.com/knadh/koanf) - Configuration management library which parses command line arguments, Go structs + creates our main configuration files

We also leverage many secondary technologies in use, including (but not limited to!):

1. [taskfile](https://taskfile.dev/usage/) - So much better than Make zomg
1. [redis](https://redis.io/) - in-memory datastore used for sessions, caching
1. databases:
* [postgres](https://www.postgresql.org/)
* [libsql](https://turso.tech/libsql)
* [sqlite](https://www.sqlite.org/)
1. [golangci-lint](https://github.com/golangci/golangci-lint) - an annoyingly opinionated linter
1. [buildkite](https://buildkite.com/theopenlane) - our CI system of choice (with github actions providing some intermediary support)

All of these components are bundled into our respective Docker images; for additional information / instructions, see the [contributing guide](.github/CONTRIBUTING.md) in this repository. We're constantly adding and changing things, but have tried to list all the great open source tools and projects we rely on; if you see your project (or one you use) in here and wish to list it, feel free to open a PR!
11 changes: 6 additions & 5 deletions query/contact.graphql
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@

mutation CreateBulkCSVContact($input: Upload!) {
createBulkCSVContact(input: $input) {
mutation CreateBulkContact($input: [CreateContactInput!]) {
createBulkContact(input: $input) {
contacts {
address
company
Expand All @@ -20,8 +19,8 @@ mutation CreateBulkCSVContact($input: Upload!) {
}
}

mutation CreateBulkContact($input: [CreateContactInput!]) {
createBulkContact(input: $input) {
mutation CreateBulkCSVContact($input: Upload!) {
createBulkCSVContact(input: $input) {
contacts {
address
company
Expand Down Expand Up @@ -90,6 +89,7 @@ query GetAllContacts {
}
}
}

query GetContactByID($contactId: ID!) {
contact(id: $contactId) {
address
Expand Down Expand Up @@ -131,6 +131,7 @@ query GetContacts($where: ContactWhereInput) {
}
}
}

mutation UpdateContact($updateContactId: ID!, $input: UpdateContactInput!) {
updateContact(id: $updateContactId, input: $input) {
contact {
Expand Down
2 changes: 0 additions & 2 deletions query/documentdatahistory.graphql
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@


query GetAllDocumentDataHistories {
documentDataHistories {
edges {
Expand Down
Loading