Potassium

The code you run on GPUs

Potassium is an open-source HTTP serving framework built for AI, sponsored by Banana.

Quickstart:

Installation

The fastest way to get up and running is to use the Banana CLI, which downloads and runs your first model.

1. Install the CLI with pip

pip3 install banana-cli

2. Create a new project directory with

banana init my-app
cd my-app

3. Start the dev server

python3 app.py

4. Call your local API (from a separate terminal)

curl -X POST -H "Content-Type: application/json" -d '{"prompt": "Hello I am a [MASK] model."}' http://localhost:8000/

🎉 Congrats! You've ran a local inference on your Potassium backend, with a BERT model

Making it your own

app.py is the core of Potassium, and where you'll be adding your custom code and models.

Read more about it and Potassium here:

pageapp.py

The banana cli also provides these files:

5. requirements.txt - python package management

6. download.py - a script ran at build-time to download models

7. Dockerfile - build steps, generally can be ignored unless you know what you're doing

Deploying

1. Push your Potassium app to Github with the include Dockerfile. It must be at the root of the repo, for now. (monorepo support coming soon)

2. Click "+ New MODEL" in the banana app, select your new repo, and deploy

Next, can then call your BERT hello world model from the SDKs, using the below json as model_inputs:

{
    'prompt': 'Hello World! I am a [MASK] machine learning model.'
}

And the SDK will return outputs such as:

{
    // some metadata
    ...
    // some metadata
    "modelOutputs": [
      [
        {
          "score": 0.0529061034321785,
          "token": 3722,
          "token_str": "simple",
          "sequence": "hello world! i am a simple machine learning model."
        },
        {
          "score": 0.050797536969184875,
          "token": 3143,
          "token_str": "complete",
          "sequence": "hello world! i am a complete machine learning model."
        },
      ]
    ]
}

Don't want to use Potassium? You can deploy any Docker workload onto Banana GPUs. Follow this guide to go beyond the Potassium framework and build your own Banana-compatible repo from scratch.