Repository source for agents

Repository source is the committed Git tree Tokay builds and deploys. It is separate from files created by the running Service or kept in Saved storage.

Last updated

AI agent? Start with llms.txt.

REPO_READ grants the complete committed tree, including binaries, generated files, dotenv files, and any secrets committed by the user. GraphQL describes the committed tree at an exact Git snapshot, while authenticated HTTP requests transfer the file contents.

For git transport access, read the remote URLs from Repo.gitSshUrl and Repo.gitHttpUrl. Never construct git server URLs from ids or hostnames yourself. gitHttpUrl carries no credentials. HTTP Basic auth is x-access-token:<token>. Without a Git credential, create one for every repository at once with issuePlatformCredential: credentialType: GIT_HTTPS_TOKEN and unpinned REPO_READ plus REPO_WRITE grants. Recommended includes CREDENTIAL_CREATE_GIT. Keep the token in git's credential helper and reuse it for every repository: the helper keeps one login per host, and every Tokay remote shares one, so a token per repository would replace the last. The token is revoked when the connection that created it disconnects or a permission step-up replaces it, so create a new one when Git rejects it.

Choose how code arrives

Ask the user whether the code is on GitHub. If it is, they link it in Tokay (below), and you import it, so GitHub stays the one source of truth. Otherwise create a repository with createRepo and push with Git; when the code is not a git repository yet and you have a shell, run git init and commit it first. Push files over HTTP or MCP push_code only without a shell or git. Keep the same way for later edits.

Branches and authority

Use createRepo for hosted source, whether the first commit comes from paste, HTTP, or Git. The first accepted branch establishes its default. Adding the remote and running git push tokay HEAD from the local repository pushes the current branch under its own name without changing the existing upstream. Hosted Git accepts ordinary heads/tags, force pushes, and deletes.

An externalGitLink makes GitHub authoritative. Tokay syncs only the default, service-selected branches, and branches awaiting explicit analysis; its clone is not a full upstream mirror. Services choose branchName in ensureServiceBundle. setServiceBranch changes it only between affected jobs and reports SERVICE_SOURCE_BUSY otherwise. The running deployment stays up; deleting the selected branch blocks new forward deployment.

For a selected branch, query sourceBranch(branchName: "...") { name headCommitHash currentSourceSnapshot { id commitHash } } on Repo. Repo.currentSourceSnapshot is the default branch only. Keep the loaded branch/base unchanged while editing; polling must not silently retarget a draft.

GitHub

GitHub access belongs to a person: Workspace.githubConnections lists the Tokay GitHub App installations and personal GitHub accounts the viewer connected, each with an opaque connection value the other operations take. githubRepos(connection:) on the same workspace asks GitHub which repositories that connection can import, so read it for one connection at a time.

  • Import with importGithubRepo(input: { workspace, connection, fullName }). It creates Code that follows GitHub and syncs it; analysis starts when the first sync lands. Optional branchName, name, and description default to the repository's default branch, name, and none.
  • Connect existing Code with connectRepoToGithub(input: { repo, connection, fullName }). Hosted Code connects only when GitHub already holds the latest commit of every branch Tokay tracks; otherwise GITHUB_CONNECT_UNPUSHED_COMMITS names the branches to push to GitHub first, so nothing made in Tokay is overwritten. Code that already follows a repository reconnects to that same repository, found by its GitHub ID, through another connection that can see it; fullName is ignored.
  • Sync now with syncExternalGitLink(input: { externalGitLink }) instead of waiting for a webhook or the next poll. Follow externalGitLink { syncState syncFailure { action code message } }.
  • Disconnect with disconnectExternalGitLink(input: { externalGitLink }). It answers CONFIRMATION_REQUIRED with its impact first; repeat the call with the token. The Code keeps the source Tokay has and becomes hosted; a pull request setup is dropped.

Setting up access needs the person in a browser signed in to Tokay. Start every setup with createGithubSetup(input: { type, workspace }):

  • type: APP_INSTALL installs or finds Tokay's GitHub App, the access a team shares. With externalGitLink it moves connected Code from a personal account to the App.
  • type: OAUTH connects the person's own account, for repositories they can read but cannot install the App on.
  • type: CODE_INSTALL with externalGitLink enables pull requests and needs a GitHub admin of the repository.

Each returns a setupUrl that expires in fifteen minutes. Hand it to the person, wait until they say they are done, then read githubConnections or the Code's externalGitLink again. GitHub returns to the webapp, which finishes setup with completeGithubSetup or cancelGithubSetup. Agents never call those. Leave returnPath out. It names a webapp page, and without it setup ends on the GitHub repository picker, or on the Code's page for a setup that served connected Code. A Bot gets PERSON_REQUIRED.

A setup cannot add a repository to an App installation Tokay already has. When githubRepos lacks the repository and an App connection's accountLogin is its owner, hand the person that connection's repositoryAccessUrl, the App's page on GitHub, where they give the App the repository. For any other owner, start an APP_INSTALL setup, which finds or installs the App on that account. A personal connection's repositoryAccessUrl is where the person asks an organization to approve Tokay. Either way, wait until they say they are done, then read githubConnections and githubRepos again.

Discover the current snapshot

query RepoSource($repoId: ID!, $after: Cursor) {
  repo(id: $repoId) {
    id
    sourceBranch { name }
    externalGitLink { id }
    viewerCanReadSource
    currentSourceSnapshot {
      id
      commitHash
      treeOid
      entryCount
      totalBytes
      sourceWarnings { code message }
      entries(prefix: "", depth: 1, first: 1000, after: $after) {
        nodes { path parentPath name type isExecutable oid sizeBytes }
        pageInfo { hasNextPage endCursor }
      }
    }
  }
}

snapshot.id is a Relay node ID. Keep it and commitHash with your analysis. Page until hasNextPage is false, passing the returned endCursor unchanged. A cursor is bound to its snapshot, prefix, and depth and cannot be reused for another query.

prefix: "", depth: 1 lists the root. To explore a directory, pass its exact path as prefix. Increase depth for bounded recursive exploration. Directories are derived metadata rows. isExecutable carries the only public Git mode distinction for a regular file. REGULAR and SYMLINK entries have byte bodies, and a symlink body is its target text. GITLINK identifies a submodule commit and has no file body.

sourceWarnings may contain GIT_LFS_OBJECTS_UNAVAILABLE. Tokay exposes committed LFS pointer blobs, not the LFS objects the provider hosts. The repository is still usable, but builds receive those pointer bytes.

COMMITTED_ENV_FILE names committed dotenv files such as .env (template files like .env.example are not counted). The build ships them, so the app runs with their values, and anyone who can read the code can read them. Move each secret to a Project Secret with setProjectSecretValueByName, then remove the file in the next push. Tokay reads only the file names, never the values.

Search a snapshot

query SearchSource($snapshotId: ID!) {
  node(id: $snapshotId) {
    ... on RepoSourceSnapshot {
      search(query: "createServer", mode: TEXT, prefix: "src", first: 50) {
        matches { path line text type sizeBytes }
        isTruncated
      }
    }
  }
}

query is a literal string of 1 to 200 characters on one line, not a pattern. TEXT (the default) searches file contents, skips binary files, and returns path, line, and up to 500 characters of text. PATH matches a case-insensitive fragment of the repository path and returns path, type, and sizeBytes. prefix limits either mode to one directory, and first is 1 to 100 (default 50). isTruncated means more matches existed; narrow the query or prefix instead of paging.

Fetch bytes

Send the same bearer token used for GraphQL:

GET /repo_source_snapshots/<snapshot.id>/content?path=src%2Fmain.py
Authorization: Bearer <token>

The response is an attachment with sniffing disabled. It streams the exact indexed Git blob and supports text, binaries, and blobs up to 100 MB. A gitlink returns SOURCE_CONTENT_UNAVAILABLE.

Fetch a whole repository or directory in one request:

GET /repo_source_snapshots/<snapshot.id>/archive?path=src&format=tar.gz
Authorization: Bearer <token>

path may be empty for the root, and format is tar.gz or zip. Tar preserves executable bits and symlinks most faithfully. ZIP is convenient for browsers, but extraction tools vary in POSIX and symlink fidelity. Archives preserve committed names, so extract untrusted repositories into an isolated workspace. A name or symlink target that is legal in Git but cannot be represented safely fails the whole request with a typed ARCHIVE_* error before headers instead of being renamed or omitted.

Normal fast forward pushes usually leave old objects reachable. A force push followed by Git garbage collection can expire a snapshot. On SNAPSHOT_EXPIRED, resolve currentSourceSnapshot again. Do not retry the old snapshot.

Download in a browser

Mint a download ticket while authenticated, then navigate to the returned path:

mutation DownloadSource($snapshot: ID!) {
  createRepoSourceDownloadTicket(input: { repoSourceSnapshot: $snapshot, format: ZIP, path: "assets" }) {
    repoSourceDownloadTicket { downloadUrl expiresAt }
  }
}

format is required and is RAW, TAR_GZ, or ZIP, and RAW requires a file path. The returned absolute downloadUrl needs no bearer token because the opaque ticket is the authorization boundary. The ticket lasts five minutes, works once, and is burned when download begins.

Submit source changes

Writes are guarded by an expected commit. Small UTF-8 text can be sent inline. Stage binary or large content first. Request an upload ticket, then PUT the raw stream to its path without a bearer:

mutation StageSource($repo: ID!) {
  createRepoSourceUploadTicket(input: { repo: $repo }) {
    repoSourceUploadTicket { uploadUrl maxBytes expiresAt }
  }
}
PUT <uploadUrl>
Content-Type: application/octet-stream
Content-Length: <bytes>

<raw bytes>

HTTP 201 returns an opaque stagedBlobHandle, SHA-256, sizeBytes, and expiry. The ticket expires after five minutes and takes one PUT. A replayed or expired ticket returns REPOSITORY_UPLOAD_TICKET_INVALID, so request a new one to retry a failed upload. A staged blob is bound to the principal and repository, expires after one hour, and can be retried after a push conflict. The repository must already exist, so create it before staging bytes for its first commit.

Commit text and staged bytes together:

POST /repos/<repo.id>/push_bundle
Authorization: Bearer <token>
Content-Type: application/json

{
  "idempotencyKey": "<unique key for this push>",
  "branchName": "<selected branch>",
  "expectedCommitHash": "<snapshot.commitHash>",
  "upsertFiles": [
    { "path": "src/main.js", "content": "console.log('hi')\n" },
    { "path": "assets/logo.png", "stagedBlobHandle": "<opaque handle>" }
  ],
  "deletePaths": ["old.txt"]
}

Every upsert has exactly one of content or stagedBlobHandle. Deletes are exact file paths. Omitted paths are preserved. For the first commit to a new hosted repo, send expectedCommitHash: null.

Apply a reviewed source-change proposal by naming it instead of reconstructing its file operations:

POST /source_change_proposals/<sourceChangeProposal.id>/push_bundle
Authorization: Bearer <token>
Content-Type: application/json

{
  "idempotencyKey": "<unique key for this apply>"
}

Proposal mode accepts only the idempotency key. Tokay resolves the reviewed proposal's repository, branch, base commit, and database-owned patch, then records the resulting commit or PR on the proposal without adding proposal metadata to Git.

Replace the whole tree by staging a tar.gz of the complete repository and naming it instead of file operations:

POST /repos/<repo.id>/push_bundle
Authorization: Bearer <token>
Content-Type: application/json

{
  "idempotencyKey": "<unique key for this push>",
  "branchName": "<selected branch>",
  "expectedCommitHash": "<snapshot.commitHash>",
  "archiveStagedBlobHandle": "<opaque handle>"
}

The archive becomes the tree, so every path it omits is deleted. That is why the mode is explicit and requires the exact current commit, or null for an empty repository. Never infer replacement from an ordinary archive or a sparse push. MCP push_code takes the same archiveStagedBlobHandle.

Both API and MCP pushes use the same durable source operation. A completed API push returns operationStatus: READY and result { commitHash, hasChanges }. A pending push returns HTTP 202 with operationStatus: PENDING, operationId, and expiresAt. Continue through GraphQL.

query SourcePush($id: ID!) {
  asyncOperation(id: $id) {
    status
    sourcePushResult { commitHash hasChanges branchName pullRequestUrl pullRequestNumber }
    failure { action code message supportRef }
  }
}

Retain the exact request, including staged handles and idempotencyKey. A lost response is recovered by resending it with the same credential and key. Results are retained for 24 hours. A changed request must use a new key, and reusing an old key with different input returns IDEMPOTENCY_KEY_REUSED. Accepted uploads remain reserved until the push is recorded or fails. A FAILED operation with action RETRY permits a new operation. On EXPIRED, check current source before creating new work.

READY means the submission completed. If pullRequestUrl is present, report it and wait for upstream merge; the base branch has not changed and no deployment is implied. Otherwise the source was committed to Tokay. For initial service selection, poll repository analysis for the accepted commit. For updates to existing services, follow the existing project’s deployReadiness through build and deployment. That pipeline does not require a new repository-analysis row. Do not recreate existing services. Request repository analysis explicitly when you need candidates for additional services.

If repository analysis fails, call requestRepoAnalysis(input: { repo: repoId, commitHash }) and resume analysis polling without pushing again.

On a FAILED operation with failure.code = GIT_PUSH_CONFLICT, refresh the current snapshot, reconcile your operations, and retry. A staged blob used by the failed attempt is still available. Hosted repositories accept both Git and browser/API commits. GitHub-connected submissions create upstream PRs on a tokay/<summary>-<id> branch, preferring Tokay's pull request App when a repository admin has enabled pull requests, and otherwise the submitting person's GitHub account when it can push. Missing write access means enabling pull requests or connecting a GitHub account that can push. It never silently changes the submission's identity. Tokay deletes the branches of App-opened PRs after they close. Workflow-file writes are unsupported.

Shell scripts

Use stack SHELL for Bash or POSIX shell scripts and select SCHEDULED, BACKGROUND, or FUNCTION. Use the script's repository path as entryPoint. Tokay invokes Bash by default, so no executable bit or dependency manifest is required. Source-backed explicit sh or bash process commands preserve their arguments and working directory.

A Shell FUNCTION runs once per request in the service root. The exact body arrives on stdin. REQUEST_METHOD, PATH_INFO, QUERY_STRING, REQUEST_URI, SERVER_PROTOCOL, REMOTE_ADDR, HTTP_HOST, CONTENT_TYPE, and CONTENT_LENGTH describe the request. Headers become uppercase HTTP_* variables with hyphens replaced by underscores. An incoming Proxy header never sets HTTP_PROXY.

stdout becomes the response body after the script exits. Exit 0 returns HTTP 200 and any other exit returns 500. Valid JSON gets application/json, otherwise text/plain. stderr goes to container logs. Do not print CGI response headers. The runner provides no custom status or header protocol.

Requests can run concurrently. Each gets a separate TMPDIR, while source and Saved storage remain shared. The runner spools input and output to temporary files, enforces a 300-second deadline, and kills the process group on timeout or disconnect. Background child processes do not survive the invocation. The build check validates script location and Bash syntax without executing the script.

The common toolbox includes curl, jq, Mike Farah's Go yq v4, archive and transfer tools, and PostgreSQL, MySQL-compatible, and Redis clients. Mongo shell and database tools are included when source invokes them, including for external connections. Connection variables use the ordinary resource, Config, and Secret delivery model.

Limits and stop conditions

  • Blob, raw read, and staged blob: 100 MB each.
  • Committed tree: 50,000 entries, 5 GB declared bytes, and 16 MB canonical metadata.
  • Sparse change: 10,000 paths and 1 GB. Inline text is 1 MB per file and 50 MB total.
  • Archive: 50,000 entries and 1 GB declared content.

A failure fails the whole operation. Tokay never silently omits paths. Treat authorization and unsupported path errors as terminal until input or grants change. Retry GIT_PUSH_CONFLICT only after refreshing. Refresh on SNAPSHOT_EXPIRED. Respect 429 concurrency responses before retrying.

If viewerCanReadSource is false, stop and report missing REPO_READ. If it is true but currentSourceSnapshot is null, the repository has no successfully ingested commit yet.