Skip to content

superglue translates data from external systems into exactly the format you need.

License

Notifications You must be signed in to change notification settings

superglue-ai/superglue

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

77 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

superglue_logo_white

data that speaks your language ๐Ÿฏ

superglue translates data from external systems into exactly the format you need. Itโ€™s an open source proxy that automatically extracts, maps and transforms data so developers donโ€™t have to write and maintain complex integration code.

  • ๐Ÿ”ฎ One-off LLM-powered translations: Generate deterministic, high-performance translation code.
  • ๐Ÿฉน Self-healing: Detect format changes and update translations automatically.
  • ๐Ÿš€ Fast deployment: Supports most APIs and data formats out of the box.

GitHub Y Combinator Client SDK Docker Twitter Adina Twitter Stefan

flowchart LR
    subgraph Input[data sources]
        A1[APIs]
        A2[files]
        A3[legacy systems]
    end

    subgraph Process[data transformation]
        T1[superglue engine]
    end

    subgraph Output[destination]
        D1[your system]
    end

    Input --> Process
    Process --> Output

    %% Styling
    classDef sourceStyle fill:#f9f,stroke:#333,stroke-width:2px
    classDef processStyle fill:#bbf,stroke:#333,stroke-width:2px
    classDef outputStyle fill:#bfb,stroke:#333,stroke-width:2px

    class Input sourceStyle
    class Process processStyle
    class Output outputStyle
Loading

quick start

hosted version

  1. Sign up for early access to the hosted version of superglue at superglue.cloud

  2. Install the superglue js/ts client:

npm install @superglue/client
  1. Configure your first api call:
import { SuperglueClient } from "@superglue/client";

const superglue = new SuperglueClient({
  apiKey: "************"
});

const config = {
  urlHost: "https://futuramaapi.com",
  urlPath: "/graphql",
  instruction: "get all characters from the show",
  responseSchema: {
    type: "object",
    properties: {
      characters: {
        type: "array",  
        items: {
          type: "object",
          properties: {
            name: { type: "string" },
            species: { type: "string", description: "lowercased" }
          }
        }
      }
    }
  }
};

const result = await superglue.call({endpoint: config});
console.log(JSON.stringify(result.data, null, 2));

/*
output:
{
  "characters": [
    {
      "name": "Phillip J. Fry",
      "species": "human"
    },
    ...
  ]
}
*/

self-hosted version

Run your own instance of superglue using Docker:

  1. Pull the Docker image:
docker pull superglueai/superglue
  1. Create a .env file with the following configuration:
# Server Configuration

# Port to run the superglue server
GRAPHQL_PORT=3000

# Port to run the web dashboard 
WEB_PORT=3001

# Endpoint the web interface will connect to
GRAPHQL_ENDPOINT=http://localhost:3000

# Authentication token for API access
AUTH_TOKEN=your-auth-token

# Datastore Configuration. Memory is faster but not persistent. Redis is slower but persistent.
DATASTORE_TYPE=redis or memory
# if redis
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_USERNAME=default
REDIS_PASSWORD=secret

# OpenAI Configuration
OPENAI_API_KEY=sk-...
# OpenAI model to use. We recommend gpt-4o-2024-11-20
OPENAI_MODEL=gpt-4o-2024-11-20
  1. Start the server:
docker run -d \
  --name superglue \
  --env-file .env \
  -p 3000:3000 \
  -p 3001:3001 \
  superglueai/superglue
  1. Verify the installation:
curl http://localhost:3000/health
> OK

# or open http://localhost:3000/?token=your-auth-token
  1. Open the dashboard to create your first configuration:
http://localhost:3001/
  1. run your first call:
npm install @superglue/client
import { SuperglueClient } from "@superglue/client";

const superglue = new SuperglueClient({
  endpoint: "http://localhost:3000",
  apiKey: "your-auth-token"
});

// either via config object
const config = {
  urlHost: "https://futuramaapi.com",
  urlPath: "/graphql",
  instruction: "get all characters from the show",
};

const result = await superglue.call({endpoint: config});

// or via the api id if you have already created the endpoint
const result2 = await superglue.call({id: "futurama-api"});

console.log(JSON.stringify(result.data, null, 2));

key features

  • LLM-Powered Data Mapping: Automatically generate data transformations using large language models
  • API Proxy: Intercept and transform API responses in real-time with minimal added latency
  • File Processing: Handle various file formats (CSV, JSON, XML) with automatic decompression
  • Schema Validation: Ensure data compliance with your specified schemas
  • Flexible Authentication: Support for various auth methods including header auth, api keys, oauth, and more
  • Smart Pagination: Handle different pagination styles automatically
  • Caching & Retry Logic: Built-in caching and configurable retry strategies

๐Ÿ“– Documentation

For detailed documentation, visit docs.superglue.cloud.

๐Ÿค contributing

We love contributions! Feel free to open issues for bugs or feature requests.

license

superglue is GPL licensed. The superglue client SDKs are MIT licensed. See LICENSE for details.

๐Ÿ™‹โ€โ™‚๏ธ support

Twitter

About

superglue translates data from external systems into exactly the format you need.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •  

Languages