Cover image for Trusted publishing to conda channels
Written on

Trusted publishing to conda channels

Wolf Vollprecht
Wolf Vollprecht

We have recently integrated support for “Trusted Publisher” configuration to the prefix.dev server. If you maintain your own channel on prefix.dev, you can now add a trusted publisher. A trusted publisher is allowed to publish to the channel without having to configure API keys! This makes it really easy to run your a conda-package-forge with your own collection of recipes. If you are familiar with the Python ecosystem, then you might know that trusted publishing has been enabled on PyPI for a while now. We are excited to bring this feature to the conda ecosystem as well.

Never forget your keys again

To configure trusted publishing, just head to your own channel and click on the “Trusted Publishers” tab. At the moment, we only support GitHub Actions as trusted publisher. On that page, enter:

  • the GitHub user- or organization-name,

  • the repository, and

  • the workflow filename that is allowed to publish to that particular channel

Press “Submit” and you’re all set! The next time the workflow runs, it will be able to publish to the channel without any further configuration.

Within the GitHub Actions workflow, you first build the conda package with rattler-build. With --skip-existing=all, we ensure that already built packages are skipped.

rattler-build build --recipe-dir . --skip-existing=all \
     --channel conda-forge --channel $YOUR_CHANNEL

The latest version of rattler-build (>=0.31.1) supports uploading with trusted publishers natively with. With the following shell script, you can gather all produced conda packages and upload them to prefix.dev.

for file in output/**/*.conda; do
    rattler-build upload prefix --channel $YOUR_CHANNEL "$file"
done

Note: in GitHub Actions the id-token: write permission is required to upload packages.

You can find an example of this on the rust-forge repository and the associated GitHub Action.

Technical details

Trusted publishing works by using the Open ID Connect token mechanism. GitHub and other cloud vendors basically publish public keys, and use their private keys to sign a payload. We can then verify that the token was indeed signed by GitHub and has the advertised properties (such as repository owner, or workflow filename). Since these are signed by asymmetric keys, we can be sure that the data is authentic and use it to allow the upload. Very neat!

In the background, rattler-build will use a GitHub API endpoint to acquire a short-lived token. The API endpoint is only available from inside the GitHub Actions runner. We then send this token to the prefix.dev server, which verifies the token and allows the upload. You can find the code here if you want to peek under the hood.

This also paves the way for more interesting integrations in the future. For example, we can now pinpoint exactly which GitHub workflow run has uploaded a given package. This data could be shown in the frontend, and you could then click on the build log to verify what was going on at build-time.

We are already planning for the next iteration of this, which is to add support for cryptographic attestations (sigstore) - and we’re working with NumFOCUS on bringing that to the conda-forge repository as well!

Run your own forge

We have already written some documentation on how to run your own forge. The idea is that you can build your own little package distribution very easily with rattler-build. An example “forge” for a few cool Rust packages that is completely automated on top of GitHub Actions lives at github.com/wolfv/rust-forge.

We would love to see more people build their own software distributions going forward, and rattler-build makes it easier than ever. If you have any questions, or would like to use trusted publishing from other cloud vendors such as GitLab or Azure, please let us know on Discord.